Editorial Type:
Article
Article Type:
Research Article

Using Deep Learning to Emulate and Accelerate a Radiative Transfer Model

Ryan Lagerquist aCooperative Institute for Research in the Atmosphere, Colorado State University, Fort Collins, Colorado
bNOAA/ESRL/GSL, Boulder, Colorado

Search for other papers by Ryan Lagerquist in
Current site
Google Scholar
PubMed Close
https://orcid.org/0000-0002-8409-415X
,
David Turner bNOAA/ESRL/GSL, Boulder, Colorado

Search for other papers by David Turner in
Current site
Google Scholar
PubMed Close
,
Imme Ebert-Uphoff aCooperative Institute for Research in the Atmosphere, Colorado State University, Fort Collins, Colorado
cDepartment of Electrical and Computer Engineering, Colorado State University, Fort Collins, Colorado

Search for other papers by Imme Ebert-Uphoff in
Current site
Google Scholar
PubMed Close
,
Jebb Stewart bNOAA/ESRL/GSL, Boulder, Colorado

Search for other papers by Jebb Stewart in
Current site
Google Scholar
PubMed Close
, and
Venita Hagerty dNOAA/ESRL/GSL/Assimilation and Verification Innovation Division, Boulder, Colorado

Search for other papers by Venita Hagerty in
Current site
Google Scholar
PubMed Close
Online Publication:
21 Sep 2021
Print Publication:
01 Oct 2021
DOI:
https://doi.org/10.1175/JTECH-D-21-0007.1
Page(s):
1673–1696
Restricted access

Abstract

This paper describes the development of U-net++ models, a type of neural network that performs deep learning, to emulate the shortwave Rapid Radiative Transfer Model (RRTM). The goal is to emulate the RRTM accurately in a small fraction of the computing time, creating a U-net++ that could be used as a parameterization in numerical weather prediction (NWP). Target variables are surface downwelling flux, top-of-atmosphere upwelling flux (FupTOA), net flux, and a profile of radiative-heating rates. We have devised several ways to make the U-net++ models knowledge-guided, recently identified as a key priority in machine learning (ML) applications to the geosciences. We conduct two experiments to find the best U-net++ configurations. In experiment 1, we train on nontropical sites and test on tropical sites, to assess extreme spatial generalization. In experiment 2, we train on sites from all regions and test on different sites from all regions, with the goal of creating the best possible model for use in NWP. The selected model from experiment 1 shows impressive skill on the tropical testing sites, except four notable deficiencies: large bias and error for heating rate in the upper stratosphere, unreliable FupTOA for profiles with single-layer liquid cloud, large heating-rate bias in the midtroposphere for profiles with multilayer liquid cloud, and negative bias at low zenith angles for all flux components and tropospheric heating rates. The selected model from experiment 2 corrects all but the first deficiency, and both models run ~104 times faster than the RRTM. Our code is available publicly.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Ryan Lagerquist, ralager@colostate.edu

Abstract

This paper describes the development of U-net++ models, a type of neural network that performs deep learning, to emulate the shortwave Rapid Radiative Transfer Model (RRTM). The goal is to emulate the RRTM accurately in a small fraction of the computing time, creating a U-net++ that could be used as a parameterization in numerical weather prediction (NWP). Target variables are surface downwelling flux, top-of-atmosphere upwelling flux (FupTOA), net flux, and a profile of radiative-heating rates. We have devised several ways to make the U-net++ models knowledge-guided, recently identified as a key priority in machine learning (ML) applications to the geosciences. We conduct two experiments to find the best U-net++ configurations. In experiment 1, we train on nontropical sites and test on tropical sites, to assess extreme spatial generalization. In experiment 2, we train on sites from all regions and test on different sites from all regions, with the goal of creating the best possible model for use in NWP. The selected model from experiment 1 shows impressive skill on the tropical testing sites, except four notable deficiencies: large bias and error for heating rate in the upper stratosphere, unreliable FupTOA for profiles with single-layer liquid cloud, large heating-rate bias in the midtroposphere for profiles with multilayer liquid cloud, and negative bias at low zenith angles for all flux components and tropospheric heating rates. The selected model from experiment 2 corrects all but the first deficiency, and both models run ~104 times faster than the RRTM. Our code is available publicly.

© 2021 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Ryan Lagerquist, ralager@colostate.edu

Supplementary Materials

    • Supplemental Materials (ZIP 24.98 MB)
Save
Cover Journal of Atmospheric and Oceanic Technology
Journal of Atmospheric and Oceanic Technology

  • Barlow, R., and H. Brunk, 1972: The isotonic regression problem and its dual. J. Amer. Stat. Assoc., 67, 140–147, https://doi.org/10.1080/01621459.1972.10481216.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Benjamin, S., and Coauthors, 2016: A North American hourly assimilation and model forecast cycle: The Rapid Refresh. Mon. Wea. Rev., 144, 1669–1694, https://doi.org/10.1175/MWR-D-15-0242.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Berthomier, L., and B. Pradel, 2021: Cloud cover nowcasting with deep learning. Conf. on Artificial Intelligence for Environmental Science, Virtual, Amer. Meteor. Soc., 12.9, https://ams.confex.com/ams/101ANNUAL/meetingapp.cgi/Paper/380983.

  • Beucler, T., M. Pritchard, S. Rasp, J. Ott, P. Baldi, and P. Gentine, 2021: Enforcing analytic constraints in neural networks emulating physical systems. Phys. Rev. Lett., 126, 098302, https://doi.org/10.1103/PhysRevLett.126.098302.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Bolton, T., and L. Zanna, 2019: Applications of deep learning to ocean data inference and subgrid parameterization. J. Adv. Model. Earth Syst., 11, 376–399, https://doi.org/10.1029/2018MS001472.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Brenowitz, N., and C. Bretherton, 2018: Prognostic validation of a neural network unified physics parameterization. Geophys. Res. Lett ., 45, 6289–6298, https://doi.org/10.1029/2018GL078510.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Brenowitz, N., T. Beucler, M. Pritchard, and C. Bretherton, 2020: Interpreting and stabilizing machine-learning parametrizations of convection. J. Atmos. Sci., 77, 4357–4375, https://doi.org/10.1175/JAS-D-20-0082.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Chen, M., X. Shi, Y. Zhang, D. Wu, and M. Guizani, 2017: Deep features learning for medical image analysis with convolutional autoencoder neural network. IEEE Trans. Big Data, 7, 750–758, https://doi.org/10.1109/TBDATA.2017.2717439.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Chen, Y., L. Bruzzone, L. Jiang, and Q. Sun, 2021: ARU-net: Reduction of atmospheric phase screen in SAR interferometry using attention-based deep residual U-net. IEEE Trans. Geosci. Remote Sens., 59, 5780–5793, https://doi.org/10.1109/TGRS.2020.3021765.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Chollet, F., 2018: Deep Learning with Python. Manning, 361.

  • Chollet, F., and Coauthors, 2020: Keras. GitHub, https://github.com/fchollet/keras.

  • Ebert-Uphoff, I., and K. Hilburn, 2020: Evaluation, tuning and interpretation of neural networks for working with images in meteorological applications. Bull. Amer. Meteor. Soc., 101, E2149–E2170, https://doi.org/10.1175/BAMS-D-20-0097.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Felt, V., S. Samsi, and M. Veillette, 2021: A comprehensive evaluation of deep neural network architectures for precipitation nowcasting. Conf. on Artificial Intelligence for Environmental Science, Virtual, Amer. Meteor. Soc., 2.4, https://ams.confex.com/ams/101ANNUAL/meetingapp.cgi/Paper/383115.

  • Fukushima, K., 1980: Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern., 36, 193–202, https://doi.org/10.1007/BF00344251.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Fukushima, K., and S. Miyake, 1982: Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position. Pattern Recognit., 15, 455–469, https://doi.org/10.1016/0031-3203(82)90024-3.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Gagne, D., S. Haupt, D. Nychka, and G. Thompson, 2019: Interpretable deep learning for spatial analysis of severe hailstorms. Mon. Wea. Rev., 147, 2827–2845, https://doi.org/10.1175/MWR-D-18-0316.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Gentine, P., M. Pritchard, S. Rasp, G. Reinaudi, and G. Yacalis, 2018: Could machine learning break the convection parameterization deadlock? Geophys. Res. Lett., 45, 5742–5751, https://doi.org/10.1029/2018GL078202.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Gil, Y., and Coauthors, 2019: Intelligent systems for geosciences: An essential research agenda. Commun. ACM, 62, 76–84, https://doi.org/10.1145/3192335.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Goodfellow, I., Y. Bengio, and A. Courville, 2016: Deep Learning. MIT Press, 781 pp., https://www.deeplearningbook.org.

  • Gupta, H., H. Kling, K. Yilmax, and G. Martinez, 2009: Decomposition of the mean squared error and NSE performance criteria: Implications for improving hydrological modelling. J. Hydrol., 377, 80–91, https://doi.org/10.1016/j.jhydrol.2009.08.003.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Hayatbini, N., A. Badrinath, W. Chapman, L. D. Monache, F. Cannon, P. Gibson, A. Subramanian, and F. Ralph, 2021: A two-stage deep learning framework to improve short range rainfall prediction. Conf. on Artificial Intelligence for Environmental Science, Virtual, Amer. Meteor. Soc., 819, https://ams.confex.com/ams/101ANNUAL/meetingapp.cgi/Paper/381949.

  • Hinton, G., N. Srivastava, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, 2012: Improving neural networks by preventing co-adaptation of feature detectors. arXiv, https://arxiv.org/abs/1207.0580.

  • Hsu, W., and A. Murphy, 1986: The attributes diagram: A geometrical framework for assessing the quality of probability forecasts. Int. J. Forecasting, 2, 285–293, https://doi.org/10.1016/0169-2070(86)90048-8.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Huang, H., and Coauthors, 2020: UNet 3+: A full-scale connected UNet for medical image segmentation. Int. Conf. on Acoustics, Speech, and Signal Processing, Barcelona, Spain, IEEE, https://doi.org/10.1109/ICASSP40776.2020.9053405.

    • Crossref
    • Export Citation

  • Iacono, M., E. Mlawer, S. Clough, and J. Morcrette, 2000: Impact of an improved longwave radiation model, RRTM, on the energy budget and thermodynamic properties of the NCAR Community Climate Model, CCM3. J. Geophys. Res., 105, 14 873–14 890, https://doi.org/10.1029/2000JD900091.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Iacono, M., E. Mlawer, J. Delamere, S. Clough, J. Morcrette, and Y. Hou, 2005: Application of the Shortwave Radiative Transfer Model, RRTMG_SW, to the National Center for Atmospheric Research and National Centers for Environmental Prediction general circulation models. Atmospheric Radiation Measurement Science Team Meeting, Daytona Beach, FL, ARM, https://www.arm.gov/publications/proceedings/conf15/extended_abs/iacono_mj.pdf.

  • Iacono, M., J. Delamere, E. Mlawer, M. Shephard, S. Clough, and W. Collins, 2008: Radiative forcing by long-lived greenhouse gases: Calculations with the AER radiative transfer models. J. Geophys. Res., 113, D13103, https://doi.org/10.1029/2008JD009944.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Krasnopolsky, V., 2020: Using machine learning for model physics: An overview. arXiv, https://arxiv.org/abs/2002.00416.

  • Krasnopolsky, V., M. Fox-Rabinovitz, Y. Hou, S. Lord, and A. Belochitski, 2010: Accurate and fast neural network emulations of model radiation for the NCEP coupled Climate Forecast System: Climate simulations and seasonal predictions. Mon. Wea. Rev., 138, 1822–1842, https://doi.org/10.1175/2009MWR3149.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Kumler-Bonfanti, C., J. Stewart, D. Hall, and M. Govett, 2020: Tropical and extratropical cyclone detection using deep learning. J. Appl. Meteor. Climatol., 59, 1971–1985, https://doi.org/10.1175/JAMC-D-20-0117.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Kurth, T., and Coauthors, 2018: Exascale deep learning for climate analytics. Int. Conf. for High Performance Computing, Networking, Storage, and Analysis, Dallas, TX, IEEE, https://doi.org/10.1109/SC.2018.00054.

    • Crossref
    • Export Citation

  • Lagerquist, R., A. McGovern, and D. Gagne, 2019: Deep learning for spatially explicit prediction of synoptic-scale fronts. Wea. Forecasting, 34, 1137–1160, https://doi.org/10.1175/WAF-D-18-0183.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lagerquist, R., J. Allen, and A. McGovern, 2020a: Climatology and variability of warm and cold fronts over North America from 1979 to 2018. J. Climate, 33, 6531–6554, https://doi.org/10.1175/JCLI-D-19-0680.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lagerquist, R., A. McGovern, C. Homeyer, D. Gagne, and T. Smith, 2020b: Deep learning on three-dimensional multiscale data for next-hour tornado prediction. Mon. Wea. Rev., 148, 2837–2861, https://doi.org/10.1175/MWR-D-19-0372.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Long, J., E. Shelhamer, and T. Darrell, 2015: Fully convolutional networks for semantic segmentation. Conf. on Computer Vision and Pattern Recognition, Boston, MA, IEEE, https://doi.org/10.1109/CVPR.2015.7298965.

    • Crossref
    • Export Citation

  • McGovern, A., R. Lagerquist, D. Gagne, G. Jergensen, K. Elmore, C. Homeyer, and T. Smith, 2019: Making the black box more transparent: Understanding the physical implications of machine learning. Bull. Amer. Meteor. Soc., 100, 2175–2199, https://doi.org/10.1175/BAMS-D-18-0195.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Mlawer, E., and D. Turner, 2016: Spectral radiation measurements and analysis in the ARM Program. The Atmospheric Radiation Measurement Program: The First 20 Years, Meteor. Monogr., No. 57, Amer. Meteor. Soc., https://doi.org/10.1175/AMSMONOGRAPHS-D-15-0027.1.

    • Crossref
    • Export Citation

  • Mlawer, E., S. Taubman, P. Brown, M. Iacono, and S. Clough, 1997: Radiative transfer for inhomogeneous atmospheres: RRTM, a validated correlated-k model for the longwave. J. Geophys. Res., 102, 16 663–16 682, https://doi.org/10.1029/97JD00237.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Morrison, H., G. de Boer, G. Feingold, J. Harrington, M. Shupe, and K. Sulia, 2012: Resilience of persistent Arctic mixed-phase clouds. Nat. Geosci., 5, 11–17, https://doi.org/10.1038/ngeo1332.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Pincus, R., and B. Stevens, 2013: Paths to accuracy for radiation parameterizations in atmospheric models. J. Adv. Model. Earth Syst., 5, 225–233, https://doi.org/10.1002/jame.20027.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Racah, E., C. Beckham, T. Maharaj, S. Kahou, Prabhat, and C. Pal, 2017: ExtremeWeather: A large-scale climate dataset for semi-supervised detection, localization, and understanding of extreme weather events. Advances in Neural Information Processing Systems, Long Beach, CA, NeurIPS, https://proceedings.neurips.cc/paper/2017/hash/519c84155964659375821f7ca576f095-Abstract.html.

  • Reichstein, M., G. Camps-Balls, B. Stevens, M. Jung, J. Denzler, and N. Carvalhais, 2019: Deep learning and process understanding for data-driven Earth system science. Nature, 566, 195–204, https://doi.org/10.1038/s41586-019-0912-1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ronneberger, O., P. Fischer, and T. Brox, 2015: U-net: Convolutional networks for biomedical image segmentation. Int. Conf. on Medical Image Computing and Computer-assisted Intervention, Munich, Germany, Technical University of Munich, https://doi.org/10.1007/978-3-319-24574-4_28.

    • Crossref
    • Export Citation

  • Sadeghi, M., P. Nguyen, K. Hsu, and S. Sorooshian, 2020: Improving near real-time precipitation estimation using a U-net convolutional neural network and geographical information. Environ. Modell. Software, 134, 104856, https://doi.org/10.1016/j.envsoft.2020.104856.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Sha, Y., D. Gagne, G. West, and R. Stull, 2020a: Deep-learning-based gridded downscaling of surface meteorological variables in complex terrain. Part I: Daily maximum and minimum 2-m temperature. J. Appl. Meteor. Climatol., 59, 2057–2073, https://doi.org/10.1175/JAMC-D-20-0057.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Sha, Y., D. Gagne, G. West, and R. Stull, 2020b: Deep-learning-based gridded downscaling of surface meteorological variables in complex terrain. Part II: Daily precipitation. J. Appl. Meteor. Climatol., 59, 2075–2092, https://doi.org/10.1175/JAMC-D-20-0058.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Shanker, M., M. Hu, and M. Hung, 1996: Effect of data standardization on neural network training. Omega, 24, 385–397, https://doi.org/10.1016/0305-0483(96)00010-2.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Stamnes, K., S. Tsay, W. Wiscombe, and K. Jayaweera, 1988: Numerically stable algorithm for discrete-ordinate-method radiative transfer in multiple scattering and emitting layered media. Appl. Opt., 27, 2502–2509, https://doi.org/10.1364/AO.27.002502.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Stewart, J., C. Kumler, D. Hall, and M. Govett, 2020: Deep learning approach for the detection of areas likely for convection initiation. Conf. on Artificial Intelligence for Environmental Science, Boston, MA, Amer. Meteor. Soc., 4.5, https://ams.confex.com/ams/2020Annual/meetingapp.cgi/Paper/365670.

  • Stone, P., 1978: Constraints on dynamical transports of energy on a spherical planet. Dyn. Atmos. Oceans, 2, 123–139, https://doi.org/10.1016/0377-0265(78)90006-4.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Turner, D. D., and Coauthors, 2004: The QME AERI LBLRTM: A closure experiment for downwelling high spectral resolution infrared radiance. J. Atmos. Sci., 61, 2657–2675, https://doi.org/10.1175/JAS3300.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Turner, D. D., M. Shupe, and A. Zwink, 2018: Characteristic atmospheric radiative heating rate profiles in Arctic clouds as observed at Barrow, Alaska. J. Appl. Meteor. Climatol., 57, 953–968, https://doi.org/10.1175/JAMC-D-17-0252.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Wallace, J., and P. Hobbs, 2006: Atmospheric Science: An Introductory Survey. Vol. 2. Elsevier, 483 pp.

  • Wang, L., K. Scott, L. Xu, and D. Clausi, 2016: Sea ice concentration estimation during melt from dual-pol SAR scenes using deep convolutional neural networks: A case study. IEEE Trans. Geosci. Remote Sens., 54, 4524–4533, https://doi.org/10.1109/TGRS.2016.2543660.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Wimmers, A., C. Velden, and J. Cossuth, 2019: Using deep learning to estimate tropical cyclone intensity from satellite passive microwave imagery. Mon. Wea. Rev., 147, 2261–2282, https://doi.org/10.1175/MWR-D-18-0391.1.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Wood, R., 2012: Stratocumulus clouds. Mon. Wea. Rev., 140, 2373–2423, https://doi.org/10.1175/MWR-D-11-00121.1.

  • Zhou, Z., M. Siddiquee, N. Tajbakhsh, and J. Liang, 2020: Unet++: Redesigning skip connections to exploit multiscale features in image segmentation. IEEE Trans. Med. Imaging, 39, 1856–1867, https://doi.org/10.1109/TMI.2019.2959609.

    • Crossref
    • Search Google Scholar
    • Export Citation
All Time Past Year Past 30 Days
Abstract Views 1092 0 0
Full Text Views 5345 3029 156
PDF Downloads 2510 623 72