• Barth, T. J., and D. C. Jespersen, 1989: The design and application of upwind schemes on unstructured meshes. Proc. AIAA 27th Aerospace Sciences Meeting, Reno, NV, American Institute of Aeronautics and Astronautics, 112, doi:10.2514/6.1989-366.

    • Crossref
    • Export Citation
  • Computational and Information Systems Laboratory, 2012: Yellowstone: IBM iDataPlex system. NCAR/UCAR. [Available online at http://n2t.net/ark:/85065/d7wd3xhc.]

  • Dawson, C., 1999: Conservative, shock-capturing transport methods with nonconservative velocity approximations. Comput. Geosci., 3, 205227, doi:10.1023/A:1011539311766.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deleersnijder, E., and G. Lebon, 2001: Enforcing the continuity equation in numerical models of geophysical fluid flows. Appl. Math. Lett., 14, 867873, doi:10.1016/S0893-9659(01)00057-X.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dennis, J., A. Fournier, W. Spotz, A. St-Cyr, M. A. Taylor, S. J. Thomas, and H. Tufo, 2005: High-resolution mesh convergence properties and parallel efficiency of a spectral element atmospheric dynamical core. Int. J. High Perform. Comput. Appl., 19, 225235, doi:10.1177/1094342005056108.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dennis, J., and Coauthors, 2012: CAM-SE: A scalable spectral element dynamical core for the Community Atmosphere Model. Int. J. High. Perform. Comput. Appl., 26, 7489, doi:10.1177/1094342011428142.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deville, M. O., P. F. Fischer, and E. H. Mund, 2002: High-Order Methods for Incompressible Fluid Flow. Cambridge Monographs on Applied and Computational Mathematics, No. 9, Cambridge University Press, 528 pp.

    • Crossref
    • Export Citation
  • Erath, C., and R. D. Nair, 2014: A conservative multi-tracer transport scheme for spectral-element spherical grids. J. Comput. Phys., 256, 118134, doi:10.1016/j.jcp.2013.08.050.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Erath, C., P. H. Lauritzen, J. H. Garcia, and H. M. Tufo, 2012: Integrating a scalable and efficient semi-Lagrangian multi-tracer transport scheme in HOMME. Procedia Comput. Sci., 9, 9941003, doi:10.1016/j.procs.2012.04.106.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Erath, C., P. H. Lauritzen, and H. M. Tufo, 2013: On mass conservation in high-order high-resolution rigorous remapping schemes on the sphere. Mon. Wea. Rev., 141, 21282133, doi:10.1175/MWR-D-13-00002.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gross, E. S., L. Bonaventura, and G. Rosatti, 2002: Consistency with continuity in conservative advection schemes for free-surface models. Int. J. Numer. Methods Fluids, 38, 307327, doi:10.1002/fld.222.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guba, O., M. Taylor, and A. St-Cyr, 2014: Optimization-based limiters for the spectral element method. J. Comput. Phys., 267, 176195, doi:10.1016/j.jcp.2014.02.029.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hall, D. M., P. A. Ullrich, K. A. Reed, C. Jablonowski, R. D. Nair, and H. M. Tufo, 2016: Dynamical Core Model Intercomparison Project (DCMIP) tracer transport test results for CAM-SE. Quart. J. Roy. Meteor. Soc., 142, 16721684, doi:10.1002/qj.2761.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Harris, L. M., P. H. Lauritzen, and R. Mittal, 2011: A flux-form version of the Conservative Semi-Lagrangian Multi-tracer transport scheme (CSLAM) on the cubed sphere grid. J. Comput. Phys., 230, 12151237, doi:10.1016/j.jcp.2010.11.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jablonowski, C., and D. L. Williamson, 2006: A baroclinic instability test case for atmospheric model dynamical cores. Quart. J. Roy. Meteor. Soc., 132, 29432975, doi:10.1256/qj.06.12.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kuo, H.-C., and R. T. Williams, 1990: Semi-Lagrangian solutions to the inviscid Burgers equation. Mon. Wea. Rev., 118, 12781288, doi:10.1175/1520-0493(1990)118<1278:SLSTTI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., and J. Thuburn, 2012: Evaluating advection/transport schemes using interrelated tracers, scatter plots and numerical mixing diagnostics. Quart. J. Roy. Meteor. Soc., 138, 906918, doi:10.1002/qj.986.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., C. Jablonowski, M. A. Taylor, and R. D. Nair, 2010a: Rotated versions of the Jablonowski steady-state and baroclinic wave test cases: A dynamical core intercomparison. J. Adv. Model. Earth Syst., 2, 15, doi:10.3894/JAMES.2010.2.15.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., R. D. Nair, and P. A. Ullrich, 2010b: A Conservative Semi-Lagrangian Multi-tracer transport scheme (CSLAM) on the cubed-sphere grid. J. Comput. Phys., 229, 14011424, doi:10.1016/j.jcp.2009.10.036.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., P. A. Ullrich, and R. D. Nair, 2011: Atmospheric transport schemes: Desirable properties and a semi-Lagrangian view on finite-volume discretizations. Numerical Techniques for Global Atmospheric Models, P. H. Lauritzen et al., Eds., Lecture Notes in Computational Science and Engineering, Vol. 80, Springer, 185–250, doi:10.1007/978-3-642-11640-7_8.

    • Crossref
    • Export Citation
  • Lauritzen, P. H., W. C. Skamarock, M. J. Prather, and M. A. Taylor, 2012: A standard test case suite for two-dimensional linear transport on the sphere. Geosci. Model Dev., 5, 887901, doi:10.5194/gmd-5-887-2012.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., J. T. Bacmeister, T. Dubos, S. Lebonnois, and M. A. Taylor, 2014a: Held–Suarez simulations with the Community Atmosphere Model spectral element (CAM-SE) dynamical core: A global axial angular momentum analysis using Eulerian and floating Lagrangian vertical coordinates. J. Adv. Model. Earth Syst., 6, 129140, doi:10.1002/2013MS000268.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., and Coauthors, 2014b: A standard test case suite for two-dimensional linear transport on the sphere: Results from a collection of state-of-the-art schemes. Geosci. Model Dev., 7, 105145, doi:10.5194/gmd-7-105-2014.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., A. J. Conley, J.-F. Lamarque, F. Vitt, and M. A. Taylor, 2015: The terminator “toy” chemistry test: A simple tool to assess errors in transport schemes. Geosci. Model Dev., 8, 12991313, doi:10.5194/gmd-8-1299-2015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lin, S.-J., 2004: A “vertically Lagrangian” finite-volume dynamical core for global models. Mon. Wea. Rev., 132, 22932307, doi:10.1175/1520-0493(2004)132<2293:AVLFDC>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lin, S.-J., and R. B. Rood, 1996: Multidimensional flux-form semi-Lagrangian transport schemes. Mon. Wea. Rev., 124, 20462070, doi:10.1175/1520-0493(1996)124<2046:MFFSLT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lopez, N. A., and M. A. Taylor, 2015: Improving the tracer correlation problem in a spectral element dynamical core. Center for Computing Research Summer Proc. 2015, Albuquerque, NM, Sandia National Laboratories, 106–117. [Available online at https://cfwebprod.sandia.gov/cfdocs/CompResearch/docs/proceedings/ccr15.pdf#page=112.]

  • Nair, R. D., and P. H. Lauritzen, 2010: A class of deformational flow test cases for linear transport problems on the sphere. J. Comput. Phys., 229, 88688887, doi:10.1016/j.jcp.2010.08.014.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nair, R. D., J. S. Scroggs, and F. H. M. Semazzi, 2002: Efficient conservative global transport schemes for climate and atmospheric chemistry models. Mon. Wea. Rev., 130, 20592073, doi:10.1175/1520-0493(2002)130<2059:ECGTSF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nair, R. D., H.-W. Choi, and H. M. Tufo, 2009: Computational aspects of a scalable high-order discontinuous Galerkin atmospheric dynamical core. Comput. Fluids, 38, 309319, doi:10.1016/j.compfluid.2008.04.006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Neale, R. B., and Coauthors, 2012: Description of the NCAR Community Atmosphere Model (CAM 5.0). NCAR Tech. Note NCAR/TN-486+STR, 274 pp. [Available online at http://www.cesm.ucar.edu/models/cesm1.0/cam/docs/description/cam5_desc.pdf.]

  • Pudykiewicz, J., R. Benoit, and A. Staniforth, 1985: Preliminary results from a partial LRTAP model based on an existing meteorological forecast model. Atmos.–Ocean, 23, 267303, doi:10.1080/07055900.1985.9649229.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Putman, W. M., and S.-J. Lin, 2007: Finite-volume transport on various cubed-sphere grids. J. Comput. Phys., 227, 5578, doi:10.1016/j.jcp.2007.07.022.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rančić, M., R. J. Purser, and F. Mesinger, 1996: A global shallow-water model using an expanded spherical cube: Gnomonic versus conformal coordinates. Quart. J. Roy. Meteor. Soc., 122, 959982, doi:10.1002/qj.49712253209.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rood, R. B., 1987: Numerical advection algorithms and their role in atmospheric transport and chemistry models. Rev. Geophys., 25, 71100, doi:10.1029/RG025i001p00071.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shashkin, V. V., and M. A. Tolstykh, 2016: Parallel implementation of the cascade mass-conserving semi-Lagrangian transport scheme. Russ. J. Numer. Anal. Math. Modell., 31, 1728, doi:10.1515/rnam-2016-0002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Starr, V. P., 1945: A quasi-Lagrangian system of hydrodynamical equations. J. Meteor., 2, 227237, doi:10.1175/1520-0469(1945)002<0227:AQLSOH>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • St-Cyr, A., C. Jablonowski, J. M. Dennis, H. M. Tufo, and S. J. Thomas, 2008: A comparison of two shallow-water models with nonconforming adaptive grids. Mon. Wea. Rev., 136, 18981922, doi:10.1175/2007MWR2108.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Taylor, M. A., 2011: Conservation of mass and energy for the moist atmospheric primitive equations on unstructured grids. Numerical Techniques for Global Atmospheric Models, P. H. Lauritzen et al. Eds., Lecture Notes in Computational Science and Engineering, Vol. 80, Springer, 357–380, doi:10.1007/978-3-642-11640-7_12.

    • Crossref
    • Export Citation
  • Taylor, M. A., and A. Fournier, 2010: A compatible and conservative spectral element method on unstructured grids. J. Comput. Phys., 229, 58795895, doi:10.1016/j.jcp.2010.04.008.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Taylor, M. A., J. Tribbia, and M. Iskandarani, 1997: The spectral element method for the shallow water equations on the sphere. J. Comput. Phys., 130, 92108, doi:10.1006/jcph.1996.5554.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ullrich, P. A., P. H. Lauritzen, and C. Jablonowski, 2009: Geometrically exact conservative remapping (GECoRe): Regular latitude–longitude and cubed-sphere grids. Mon. Wea. Rev., 137, 17211741, doi:10.1175/2008MWR2817.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ullrich, P. A., T. Melvin, C. Jablonowski, and A. Staniforth, 2014: A proposed baroclinic wave test case for deep- and shallow-atmosphere dynamical cores. Quart. J. Roy. Meteor. Soc., 140, 15901602, doi:10.1002/qj.2241.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zarzycki, C. M., M. N. Levy, C. Jablonowski, J. R. Overfelt, M. A. Taylor, and P. A. Ullrich, 2014: Aquaplanet experiments using CAM’s variable-resolution dynamical core. J. Climate, 27, 54815503, doi:10.1175/JCLI-D-14-00004.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zerroukat, M., N. Wood, and A. Staniforth, 2002: SLICE: A semi-Lagrangian inherently conserving and efficient scheme for transport problems. Quart. J. Roy. Meteor. Soc., 128, 28012820, doi:10.1256/qj.02.69.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery
    Fig. 1.

    A graphical illustration of (a) the finite-volume Lagrangian discretization of the continuity equation and (b)–(e) the Eulerian finite-volume flux-form discretization. In (a), the upstream Lagrangian area (light blue shading) ends up at the regular Eulerian grid cell after one time step. The filled circles mark the vertices of the Eulerian cell , and the unfilled circles are the corresponding upstream vertices or departure points. In (b)–(e) the swept flux areas (gray shaded areas) are shown, where , respectively.

  • View in gallery
    Fig. 2.

    (a) The GLL points (filled red circles) in an element (light blue boundary). (b) The CSLAM control volumes in an element. (c) Notation for the th CSLAM cell. Sides ε, , are numbered counterclockwise around the cell. The vertices are located at gnomonic coordinates , , with counterclockwise orientation. Each cell side has an associated unit vector (dark blue arrows) pointing inwards. (d) Notation for the perpendicular swept flux area for side . The flux area has vertices , , , and (marked with filled purple circles). The large black arrow points in the direction of the SE flux (outflow through side one). The dashed arrows show the orientation of the line integrals used to compute the perpendicular swept flux.

  • View in gallery
    Fig. 3.

    Step-by-step example of the consistent SE-CSLAM algorithm for face 3. (a),(b) Compute the CSLAM flux perpendicular to each face by iteration. (c) The intersection between the upstream perpendicular flux faces defines the departure points for CSLAM (dark blue filled circles). (d) A degree of freedom for the swept flux area is introduced at the intersection between the perpendicular swept y-flux area and the coordinate line (red circle filled with yellow). (e),(f) By iteration, the degree of freedom is moved up and down along the coordinate axis until the flux area exactly matches the SE flux.

  • View in gallery
    Fig. 4.

    Perpendicular swept flux situations in which the displacement γ for the center vertex (, , , and ) is set to zero. (a) For flow situations in which the perpendicular swept fluxes follow a pattern of circular flow, the displacement is set to zero. Similarly for circular flow in the reverse direction. (b) When the perpendicular fluxes are negative (outflow) for sides in cell as well as sides in cell , then the displacement is set to zero. Similarly when for and for .

  • View in gallery
    Fig. 5.

    (a) Definition of displacement variable d for side 1 in cell : , , , , , , , and . The reason for defining and is that if there is a circular or degenerate flow case at vertex or , then the displacements and are no longer identical. (b) The associated basis/unit vectors for each side: , , , , , , , , , , and . The vertex notation is as follows: , , , , , and . For cells in the interior of a panel, and ; however, for cells at cubed-sphere corners and sides, this is no longer the case. In yellow boxes, the area index is written; that is, areas 1, 2, 3, 4, and 5 correspond to cells , , , , and , respectively, for side .

  • View in gallery
    Fig. 6.

    A depiction of the categorization of possible flow cases defined in (15). The light blue shaded area is the first-guess swept flux. The dark blue filled circles are the departure points (that remain fixed) and the yellow filled circles are moved along the coordinate lines (according to red arrows) until the swept airmass flux matches the SE mass flux to round off. The middle and right columns showcase where one or two departure points coincide with a grid line.

  • View in gallery
    Fig. 7.

    A graphical illustration of two different ways to make CSLAM and SE consistent. The dark blue filled circles are the departure points, the dark blue lines are the sides of the Lagrangian CSLAM cells, and the black lines are the CSLAM control volumes. The arrows indicate the points that are perturbed to make SE and CSLAM consistent. (a) SE and CSLAM are made consistent by perturbing the departure points. Note that this problem is a global iteration problem requiring computation of overlap areas during each iteration. (b) The approach chosen in this paper. The intersections between the sides of the Lagrangian cells with the grid lines are perturbed. This constitutes a local iteration problem as fluxes are matched with SE fluxes for each flux side, whereas the departure points stay fixed.

  • View in gallery
    Fig. 8.

    (columns from left to right) 1° SE, 1° CSLAM, and ¼° SE solutions for tracer one at (top) day 0, (middle) day 5, and (bottom) day 10 at level 20 (at approximately 610 hPa).

  • View in gallery
    Fig. 9.

    As in Fig. 8, but for tracer two and (top) day 0, (middle) day 8, and (bottom) day 10.

  • View in gallery
    Fig. 10.

    As in Fig. 8, but for tracer three . Note that the background value is nonzero and remains constant throughout the simulation; that is, the transport algorithms are consistent/free-stream preserving.

  • View in gallery
    Fig. 11.

    Standard error norm for (a) , (b) , (c) , and (d) using SE and CSLAM. The ¼° SE solution is used as the “truth” for computations.

  • View in gallery
    Fig. 12.

    Initial condition for (top left) , (top right) , (bottom left) , and (bottom right) used in the “toy” terminator chemistry test.

  • View in gallery
    Fig. 13.

    As in Fig. 12, but for day 15 using CSLAM.

  • View in gallery
    Fig. 14.

    As in Fig. 12, but for day 15 using SE.

  • View in gallery
    Fig. 15.

    (a) Time (s) spent in various parts of the CSLAM transport algorithm for a 1-day simulation using 256 tasks as a function of number of tracers (the yellow line is on top of the orange line). The throughput for SE transport based on the CESM1 release (CAM-SE) is also shown. See text for details. (b) As in (a) but for a fixed number of tracers (40) and as a function of tasks. Note that for the rightmost data point (5400 tasks), there is only one element in the horizontal per processor. This is the limit of scalability for CAM-SE and CAM-SE–CSLAM without threading.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 373 152 46
PDF Downloads 204 74 4

CAM-SE–CSLAM: Consistent Coupling of a Conservative Semi-Lagrangian Finite-Volume Method with Spectral Element Dynamics

Peter Hjort LauritzenNational Center for Atmospheric Research, Boulder, Colorado

Search for other papers by Peter Hjort Lauritzen in
Current site
Google Scholar
PubMed
Close
,
Mark A. TaylorSandia National Laboratories, Albuquerque, New Mexico

Search for other papers by Mark A. Taylor in
Current site
Google Scholar
PubMed
Close
,
James OverfeltSandia National Laboratories, Albuquerque, New Mexico

Search for other papers by James Overfelt in
Current site
Google Scholar
PubMed
Close
,
Paul A. UllrichDepartment of Land, Air, and Water Resources, University of California, Davis, Davis, California

Search for other papers by Paul A. Ullrich in
Current site
Google Scholar
PubMed
Close
,
Ramachandran D. NairNational Center for Atmospheric Research, Boulder, Colorado

Search for other papers by Ramachandran D. Nair in
Current site
Google Scholar
PubMed
Close
,
Steve GoldhaberNational Center for Atmospheric Research, Boulder, Colorado

Search for other papers by Steve Goldhaber in
Current site
Google Scholar
PubMed
Close
, and
Rory KellyNational Center for Atmospheric Research, Boulder, Colorado

Search for other papers by Rory Kelly in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

An algorithm to consistently couple a conservative semi-Lagrangian finite-volume transport scheme with a spectral element (SE) dynamical core is presented. The semi-Lagrangian finite-volume scheme is the Conservative Semi-Lagrangian Multitracer (CSLAM), and the SE dynamical core is the National Center for Atmospheric Research (NCAR)’s Community Atmosphere Model–Spectral Elements (CAM-SE). The primary motivation for coupling CSLAM with CAM-SE is to accelerate tracer transport for multitracer applications. The coupling algorithm result is an inherently mass-conservative, shape-preserving, and consistent (for a constant mixing ratio, the CSLAM solution reduces to the SE solution for air mass) transport that is efficient and accurate. This is achieved by first deriving formulas for diagnosing SE airmass flux through the CSLAM control volume faces. Thereafter, the upstream Lagrangian CSLAM areas are iteratively perturbed to match the diagnosed SE airmass flux, resulting in an equivalent upstream Lagrangian grid that spans the sphere without gaps or overlaps (without using an expensive search algorithm). This new CSLAM algorithm is not specific to airmass fluxes provided by CAM-SE but applies to any airmass fluxes that satisfy the Lipshitz criterion and for which the Courant number is less than one.

The National Center for Atmospheric Research is sponsored by the National Science Foundation.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author e-mail: Peter Hjort Lauritzen, pel@ucar.edu

Abstract

An algorithm to consistently couple a conservative semi-Lagrangian finite-volume transport scheme with a spectral element (SE) dynamical core is presented. The semi-Lagrangian finite-volume scheme is the Conservative Semi-Lagrangian Multitracer (CSLAM), and the SE dynamical core is the National Center for Atmospheric Research (NCAR)’s Community Atmosphere Model–Spectral Elements (CAM-SE). The primary motivation for coupling CSLAM with CAM-SE is to accelerate tracer transport for multitracer applications. The coupling algorithm result is an inherently mass-conservative, shape-preserving, and consistent (for a constant mixing ratio, the CSLAM solution reduces to the SE solution for air mass) transport that is efficient and accurate. This is achieved by first deriving formulas for diagnosing SE airmass flux through the CSLAM control volume faces. Thereafter, the upstream Lagrangian CSLAM areas are iteratively perturbed to match the diagnosed SE airmass flux, resulting in an equivalent upstream Lagrangian grid that spans the sphere without gaps or overlaps (without using an expensive search algorithm). This new CSLAM algorithm is not specific to airmass fluxes provided by CAM-SE but applies to any airmass fluxes that satisfy the Lipshitz criterion and for which the Courant number is less than one.

The National Center for Atmospheric Research is sponsored by the National Science Foundation.

© 2017 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author e-mail: Peter Hjort Lauritzen, pel@ucar.edu

1. Introduction

State-of-the-art atmosphere models prognose an increasing number of tracers, which involves solving the continuity equation of each tracer. Tracers in atmosphere models typically include different forms of water (humidity, cloud water, cloud ice, rain, snow, etc.). Models with a comprehensive treatment of aerosols explicitly solve a number of aerosol continuity equations, and models with comprehensive chemistry prognose many chemical species. For example, NCAR’s Community Atmosphere Model (CAM), version 5 (Neale et al. 2012), in its standard configuration solves 30 continuity equations in addition to the continuity equation for air and over 120 continuity equations when run with comprehensive chemistry. Hence, the number of continuity equations in state-of-the-art models easily outnumbers the remaining equations of motion [momentum equations and thermodynamic equation(s)]. Consequently, the computational cost of tracer transport may dominate the overall cost of the model even if tracer transport is supercycled (i.e., run with a larger time step than the dynamics).

Many tracer distributions in the atmosphere have complex spatial structures associated with them. For example, photolysis-driven chemical species exhibit large gradients near the solar terminator as they are destroyed or created by sunlight and moist processes; for example, convection can introduce grid-scale structure in the water variables. It is usually challenging for transport algorithms to accurately represent the transport process for tracer distributions with large gradients and grid-scale structure. It is important that the transport scheme is not a spurious source or sink of mass. For coupling with parameterizations, it is desirable that the transport scheme provides physically realizable solutions; in particular, it should not produce negative values for a positive definite tracer. To avoid overshoots near large gradients, it is desirable that the scheme is monotone (also loosely referred to as shape preserving or nonoscillatory). For the representation of aerosols and chemical species, it is important that the transport scheme does not spuriously perturb relations between tracers (if they are functionally related; e.g., see Lin and Rood 1996; Lauritzen and Thuburn 2012). For a more complete discussion on desirable properties for transport schemes intended for climate and climate/chemistry applications, see, for example, Lauritzen et al. (2011).

To assess the transport scheme’s ability to advect rough distributions and maintain nonlinear correlations between trace species, Lauritzen et al. (2012) created an idealized test case suite for modelers to assess desirable properties for their transport algorithm in an idealized setup. This test case suite used an idealized 2D wind field from Nair and Lauritzen (2010), with a range of initial condition distributions for tracers ranging from smooth to distributions with discontinuous gradients and a wide range of existing and new diagnostics. Results from over a dozen transport algorithms are presented in Lauritzen et al. (2014b).

To improve the scalability of CAM based on the finite-volume dynamical core (CAM-FV; Lin 2004), the spectral element (SE) dynamical core was imported into CAM from High-Order Methods Modeling Environment (HOMME; Dennis et al. 2005; Nair et al. 2009; Taylor and Fournier 2010) and is referred to as CAM-SE. CAM-SE is based on a continuous Galerkin finite-element (or spectral element) method (Taylor et al. 1997) and is discretized using a cubed-sphere tiling of the sphere. Petascale scalability of CAM-SE was demonstrated in Dennis et al. (2012). Also, CAM-SE provided refined mesh functionality in CAM (e.g., St-Cyr et al. 2008; Zarzycki et al. 2014), improved accuracy of idealized baroclinic wave simulations (Lauritzen et al. 2010a), conserved total energy to time-truncation errors (Taylor 2011), and improved global axial angular momentum budgets compared to CAM-FV (Lauritzen et al. 2014a). That said, the tracer transport component of CAM-SE appeared to be less accurate than CAM-FV for nonsmooth tracer distributions (Lauritzen et al. 2014b; Hall et al. 2016). In an idealized terminator test in which two species react according to a reaction coefficient proportional to the solar terminator while being transported by an idealized 2D flow, CAM-SE produces errors larger than CAM-FV (Lauritzen et al. 2015). The errors are due to the limiter used for tracer transport that prevents oscillatory (and, in particular, negative) solutions for tracers (Guba et al. 2014). In terms of computational throughput, it was also found that CAM-SE was slower for large tracer counts (at least when run with core counts similar to the nonscalable CAM-FV).

It is the overall purpose of this paper to improve the accuracy and efficiency of tracer transport in CAM-SE. To improve the accuracy and efficiency of multitracer transport, the Conservative Semi-Lagrangian Multitracer (CSLAM; Lauritzen et al. 2010b) transport scheme was implemented in HOMME (Erath et al. 2012, 2013). Later, an alternative inherently conservative semi-Lagrangian scheme called spectral element Lagrangian transport (SPELT; Erath and Nair 2014) was implemented in HOMME, which easily supports mesh-refinement applications and has a simpler search algorithm compared to the original CSLAM (note that the new version of CSLAM presented here does not have a comprehensive search algorithm). Both SPELT and CSLAM are fully 2D finite-volume semi-Lagrangian schemes. Semi-Lagrangian schemes based on dimensional splitting along coordinate lines (Putman and Lin 2007) or Lagrangian translations of coordinate lines (conservative cascade methods; e.g., Nair et al. 2002; Zerroukat et al. 2002; Shashkin and Tolstykh 2016) are alternatives to fully 2D methods that are more easily extensible to 3D methods. Cascade schemes are computationally efficient and designed for structured orthogonal grid systems. It is unclear how to extend the cascade methods to nonorthogonal grids—in particular, the gnomonic cubed-sphere grid used here—as the grid lines are discontinuous near the cubed-sphere edges and corners. All of these schemes are stable for long time steps, inherently mass conservative, and shape preserving; some of them preserve linear relations between tracers even with a shape-preserving limiter (e.g., CSLAM) and are efficient for large tracer counts as the upstream Lagrangian grids can be reused for each additional tracer. Specific to this study, we note that in 2D idealized test cases, the CSLAM scheme is more accurate than CAM-SE in terms of the diagnostics used in the test case suite presented in Lauritzen et al. (2012).

For 3D applications in CAM-SE, the winds for CSLAM are provided by CAM-SE, which creates an inconsistency between the continuity equation for air (solved with the SE method) and tracers (solved with CSLAM). In other words, if a tracer is uniformly one, then the solution for tracer mass, computed with CSLAM, does not reduce to the CAM-SE continuity equation for air. Inconsistency can lead to loss of tracer mass conservation and/or unphysical solutions (nonmonotone). This problem of the discretization of the transport equation not being consistent with the airmass equation has been known for some time (Rood 1987; Lin and Rood 1996; Gross et al. 2002). Discretizations that are consistent are also referred to in the literature as free-stream preserving, constancy preserving, or compatible. Solutions have been proposed for shallow-water applications in which correction terms in the velocity fields have been introduced to enforce consistency (e.g., Deleersnijder and Lebon 2001) or using a corrective mass flux in the transport scheme (e.g., Dawson 1999).

In this paper, we present a CSLAM-based algorithm in which CSLAM is made consistent with the SE continuity equation for air so that the CSLAM solution to the tracer continuity equations is mass conservative, monotone, and consistent. This is achieved by diagnosing CAM-SE airmass fluxes through CSLAM control volume edges and using that information to construct CSLAM control volumes for which the airmass flux implied by CSLAM matches CAM-SE fluxes to round off. Consistency is effectively achieved by finding an upstream departure grid for which the CSLAM solution for air mass matches the CAM-SE solution for air mass to round off. Rather than manipulating velocity components or adjusting fluxes a posteriori, the consistency is enforced in Lagrangian space. Note that the algorithm for enforcing consistency in CSLAM is general. Any airmass fluxes (so not necessarily CAM-SE diagnosed fluxes) that satisfy the Lipschitz criterion (also known as the deformational Courant number by which the upstream trajectories do not cross intersect in a single time step; Pudykiewicz et al. 1985; Kuo and Williams 1990) and for which the Courant number is less than one can be used.

The paper is organized as follows: The continuity equation for CSLAM and SE are introduced in section 2 as well as the consistent coupling problem formulation. In section 3, the consistent SE-CSLAM algorithm is presented, and in section 4, we present results from idealized baroclinic wave simulations with inert and reactive tracers in order to assess accuracy and computational efficiency. A summary is provided in section 5.

2. Methods

a. Finite-volume continuity equation

Assume a floating Lagrangian vertical coordinate that initially coincides with the hybrid-sigma coordinates (Lin 2004; Starr 1945). The air mass in each layer is , where is the pressure-level thickness (for notational simplicity, the vertical index is dropped) and is the area of the regular (Eulerian) grid cell . Similar variable definitions for the tracer mass , where Q is mixing ratio. The gnomonic coordinate locations of the vertices of associated with the cubed-sphere geometry (Rančić et al. 1996) are , , , and . After a number of time steps in which the vertical coordinate is floating, all variables are remapped back to an Eulerian reference vertical coordinate (Lin 2004).

Consider the time-discretized integral form of the continuity equation for air and tracer mass discretized in time using floating Lagrangian vertical coordinates:
e1
(see Fig. 1) where τ is the time-level index, and is the upstream Lagrangian area that, after one time step , ends up at .
Fig. 1.
Fig. 1.

A graphical illustration of (a) the finite-volume Lagrangian discretization of the continuity equation and (b)–(e) the Eulerian finite-volume flux-form discretization. In (a), the upstream Lagrangian area (light blue shading) ends up at the regular Eulerian grid cell after one time step. The filled circles mark the vertices of the Eulerian cell , and the unfilled circles are the corresponding upstream vertices or departure points. In (b)–(e) the swept flux areas (gray shaded areas) are shown, where , respectively.

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

The Lagrangian form of the continuity equation (1) can also be written in flux form (Harris et al. 2011):
e2
where the mass fluxes through the four edges of a CSLAM cell are given by
e3
and is −1 for outflow and +1 for inflow. Note that the union of all the flux areas with appropriate signs yields the Lagrangian upstream area:
e4
Hence, the two forms of the continuity equation are, as expected, equivalent in their continuous forms.

In the CSLAM discretization, the vertices of are traced upstream and connected with straight lines to form . To make the scheme third order, a subgrid-scale distribution of ψ is computed (fully 2D degree-two Lagrange polynomial; Ullrich et al. 2009) with mass conservation as a constraint; that is, the polynomial integrated over yields the gridcell average value . The integral of the polynomial over the overlap areas between and the regular grid cells is computed via line integrals. Since the scheme is fully two-dimensional, it extends to the cubed sphere quite easily in the sense that it can accurately represent the deformation of the Lagrangian cells when they span different panels and hence different projections [see, e.g., Fig. 6 in Nair and Lauritzen (2010)]. For more details on the CSLAM scheme, see Lauritzen et al. (2010b). For the coupling between tracer mass and air mass, when both are solved with the CSLAM scheme, see appendix B in Nair and Lauritzen (2010). It may be shown that even in their discretized forms, the flux-form version of CSLAM based on (2) is equivalent to (1). Monotone solutions for Q are obtained by applying a limiter/filter to the subgrid-scale reconstruction function for Q so that its extrema are within the range of the surrounding gridcell average values of Q (Barth and Jespersen 1989).

b. SE continuity equation

The SE continuity equation for air mass is based on the advection–diffusion equation
e5
where is the velocity vector, κ is the order of the viscosity, and ν is the (hyper)viscosity coefficient. For simplicity, the derivations below assume . The SE method uses a weak formulation based on globally continuous test functions and integration over the entire computational domain. A suitable choice of Lagrange basis functions and a Gauss–Lobatto–Legendre (GLL) quadrature inner product leads to a diagonal mass matrix, and the SE method can then be formulated in terms of operations local to each element, followed by the direct stiffness summation (DSS) operation to maintain continuity at the element edges. For more details, see Deville et al. (2002).
For the SE method, we define the inner product via
e6
where J is the Jacobian of the map from the quadrilateral spherical element to the reference element ; are the GLL quadrature points in each element (see Fig. 2a); and are polynomials; and and are the GLL quadrature weights [for more information on the HOMME grid system, see Nair et al. (2009)]. Note that the GLL quadrature point locations use subscript m and n, whereas the CSLAM control volume edges use subscript i and j.
Fig. 2.
Fig. 2.

(a) The GLL points (filled red circles) in an element (light blue boundary). (b) The CSLAM control volumes in an element. (c) Notation for the th CSLAM cell. Sides ε, , are numbered counterclockwise around the cell. The vertices are located at gnomonic coordinates , , with counterclockwise orientation. Each cell side has an associated unit vector (dark blue arrows) pointing inwards. (d) Notation for the perpendicular swept flux area for side . The flux area has vertices , , , and (marked with filled purple circles). The large black arrow points in the direction of the SE flux (outflow through side one). The dashed arrows show the orientation of the line integrals used to compute the perpendicular swept flux.

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

The SE advection step may be written as
e7
where ϕ is a test function, and , and is the nodal flux corresponding to the DSS operation (i.e., summation at element edges combined with multiplication of the inverse mass matrix). Note that if , then the right-hand side of (7) is the mass change within each element.
It may be shown that there exists a test function that gives the change of mass over the th CSLAM control volume ():
e8
Moreover, the change of mass in each subcell () given in (8) is given by a sum of edge mass fluxes :
e9
and for each face, there is a flux contribution from , , and . In other words, the SE airmass flux can be diagnosed and is derived in detail in the appendix. This SE airmass flux is essential for the coupling algorithm described in this paper. First, let us formulate the coupling problem.

c. Problem formulation

The challenge is to couple the CSLAM continuity equation for tracers with the SE continuity equation for air without violating the following:

  • Tracer mass conservation

  • Consistency (aka compatibility, free-stream preservation, constancy preservation)

  • Tracer shape preservation

  • Linear correlations between tracers (e.g., Lin and Rood 1996; Lauritzen and Thuburn 2012)

Here, consistency refers to the property that the continuity equation for tracer mass reduces to the equation for air mass for in discretized space. This is trivial if the tracer transport equation is solved with the same numerical method as the continuity equation for air. The consistency is violated if, for example, the continuity equation for tracers is forced with reanalysis data that prescribe winds and surface pressure or, as is the case here, if different numerical methods (on different grids) are used for the continuity equation for tracers and air mass.

If one can find an upstream Lagrangian grid that satisfies the following constraints, then the requirements listed above are automatically satisfied.

Constraint 1: the CSLAM fluxes must match the SE fluxes:
e10
Constraint 2: the upstream Lagrangian cells span the sphere without gaps and/or overlaps:
e11
where is the surface of the sphere. Note that the upstream grid constructed from using a standard semi-Lagrangian trajectory algorithm does not necessarily provide CSLAM fluxes that satisfy constraint 1 and therefore, the airmass consistency is violated. If the individual airmass fluxes from CSLAM are overwritten with SE fluxes, the tracer shape preservation is violated.

The problem of developing swept areas to match the SE fluxes may initially seem intractable, as the two methods appear very different (SE is often thought of as a pointwise method in contrast with the volumetric approach of finite-volume methods). However, local construction of the swept areas is made possible by the fact that, on the level of individual spectral elements, there is a well-defined notion of fluxes between adjacent elements. It is exactly this correspondence that allows SE methods to maintain global mass conservation on a local level.

3. Consistent CSLAM flux algorithm

The algorithm has been designed to avoid (as much as possible) conditional statements in the code for control volumes at the cubed-sphere edges. Hence, there is a substantial amount of notation that needs to be introduced. The basic algorithm is explained in Fig. 3.

Fig. 3.
Fig. 3.

Step-by-step example of the consistent SE-CSLAM algorithm for face 3. (a),(b) Compute the CSLAM flux perpendicular to each face by iteration. (c) The intersection between the upstream perpendicular flux faces defines the departure points for CSLAM (dark blue filled circles). (d) A degree of freedom for the swept flux area is introduced at the intersection between the perpendicular swept y-flux area and the coordinate line (red circle filled with yellow). (e),(f) By iteration, the degree of freedom is moved up and down along the coordinate axis until the flux area exactly matches the SE flux.

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

a. Departure point algorithm

In this section, subscript has been dropped for notational simplicity. The departure points are defined in terms of perpendicular swept areas, (Fig. 2d), for which the swept CSLAM mass flux matches the SE mass flux to round off. The mathematical formulation is as follows: for CSLAM control volume , find perpendicular swept area for side , so that
e12
where is the perpendicular swept area (a rectangle) with vertices , , , and , where is the displacement parameter (Fig. 2d), and is the unit vector perpendicular to side ε (Fig. 2c). The integral on the right-hand side of (12) is converted to line integrals using the divergence theorem, and the line integrals are computed using Gauss quadrature [for details, see Lauritzen et al. (2010b)]. Note that the potentials for the line integrals are chosen such that lines with constant x coordinates have zero line integrals.

The swept area is defined in terms of one independent variable , and the integral on the right-hand side of (12) is monotonically increasing with increasing . This constitutes a well-posed iteration problem that is solved with the Newton iteration algorithm. For notational simplicity, we have dropped the side index ε in the pseudoalgorithm below.

Let be the difference between the perpendicular CSLAM swept flux (which is a function of γ) and the SE flux :
e13
where ϑ is the iteration index. The iteration procedure is as follows:
  1. Set .

  2. Set and compute .

  3. Compute
    eq1
  4. Set and compute .

  5. If is less than a given tolerance (e.g., ), the iteration has converged; else set and and repeat steps 3–5.

  6. Done: set .

Note that when computing the line integrals for , line segments that are x isolines are zero and only the line segments that are a function of γ need to be recomputed during each Newton iteration. Hence, for sides , there are only two line segments that need to be recomputed for each iteration, and for sides , there is one. The iteration algorithm is therefore efficient.
Obviously, the fluxes are equal with opposite sign for sides shared between control volumes. Hence, the displacements, , are only computed for outflow () and set to zero for inflow. For sides located in nonexistent cells in physical space [in other words, the halo cells located at the cubed-sphere corners; Fig. 5 in Erath et al. (2012)], the displacements are set to zero. The departure point corresponding to arrival point is given by
e14
To make sure the iterative algorithm for the final swept area is well conditioned and simple, we set the displacements to zero for circular flow and “degenerate” flow conditions (described in Fig. 4). If this is not done, the iterative algorithm for the final swept areas (described in the next section) becomes more complicated and expensive. If we assume that the SE fluxes satisfy a flow-deformation criterion similar to the Lipschitz criterion (also referred to as deformational Courant number), the circular and degenerate flow cases are limited in terms of the size of the swept fluxes.
Fig. 4.
Fig. 4.

Perpendicular swept flux situations in which the displacement γ for the center vertex (, , , and ) is set to zero. (a) For flow situations in which the perpendicular swept fluxes follow a pattern of circular flow, the displacement is set to zero. Similarly for circular flow in the reverse direction. (b) When the perpendicular fluxes are negative (outflow) for sides in cell as well as sides in cell , then the displacement is set to zero. Similarly when for and for .

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

b. Final swept flux area

The algorithm for calculating the final swept areas is the same for each side with appropriate index change. Hence, we describe the algorithm only for side 1, , in cell . Define displacement variables (defined as the sum of the displacements on either side of a cell wall) and associated unit/basis vectors (defined in Fig. 5). The definitions of d and for sides 2, 3, and 4 are obtained by rotating the figure 90°, 180°, and 270°, respectively, about the center for cell . The subscript k is chosen quite arbitrarily. The purpose of introducing this notation is so that after the association of and with the relevant displacements and face unit vectors, the algorithm is the same for all faces. The notation used here closely follows the FORTRAN source code.

Fig. 5.
Fig. 5.

(a) Definition of displacement variable d for side 1 in cell : , , , , , , , and . The reason for defining and is that if there is a circular or degenerate flow case at vertex or , then the displacements and are no longer identical. (b) The associated basis/unit vectors for each side: , , , , , , , , , , and . The vertex notation is as follows: , , , , , and . For cells in the interior of a panel, and ; however, for cells at cubed-sphere corners and sides, this is no longer the case. In yellow boxes, the area index is written; that is, areas 1, 2, 3, 4, and 5 correspond to cells , , , , and , respectively, for side .

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

For the categorization of the flow cases (Fig. 6), the logical variables , , , and are defined as follows:
eq2
and similarly (by appropriate rotation) for sides 2, 3, and 4. The variable is true if a departure point is located in area r (see Fig. 5b for the area r definition). The challenges in the categorization are the pathological cases where some displacements are zero. The categorization into flow cases depicted in Fig. 6 is given by
e15
Having defined the flow cases, a well-posed and simple iteration algorithm is constructed to make the CSLAM swept fluxes identical to the prescribed SE fluxes . For that, a swept area that decreases and increases monotonically as a function of one parameter, γ, is defined. We then iteratively compute γ by minimizing the difference between the swept CSLAM flux and SE flux:
e16
where is the overlap area between the swept flux area and Eulerian cell k. For side 1, the Eulerian areas are . A well-posed iteration problem is obtained by defining the CSLAM swept areas using the following criteria:
  • If the departure point is located in area 1 or 2 (cases 1, 2, 4, 6, 7, and 8), then the first-guess crossing with side 4 is given by for the line integrals in area 3 and in area 2 (see magenta filled circles in Fig. 6). The coordinate crossings are the same if side 4 is not coinciding with a cubed-sphere panel side. Similarly, if the departure point is located in area 3 or 4 (cases 1, 3, 5, 7, and 8), then the first-guess crossing with side 2 is given by for the line integrals in area 3 and for the line integrals in area 4 (see red filled circles in Fig. 6). Note that if γ is decreased, the swept area decreases.

  • If the departure point is located in area 1, the first-guess crossing with side 1 of cell is computed as the crossing between the line connecting the departure point (blue filled circle in area 1) and the crossing computed above (magenta filled circle) given by
    e17
    • (green filled circle), where
      e18
    • and for the initial iteration. Similarly, if the departure point is located in area 5, the first-guess crossing with side 1 of cell is given by
      e19
    • (green filled circle), where
      e20
    • (cyan filled circle).

  • If both departure points, and , are located in area 3, then they are given by
    e21
    e22
    • There is no crossing between the upstream flux side and the coordinate lines. We introduce a point halfway between the departure points:
      e23
    • and it is perturbed along a vector perpendicular to the flux side—that is, a 90° counterclockwise rotation of the vector (see Fig. 6; case 0).

Defining the swept flux overlap areas as a function of γ as described above guarantees that the swept flux area is simply connected and that the iteration problem is well posed. Also, it provides an accurate approximation of flux areas across cubed-sphere panel sides. The vertices for the overlap areas are defined in terms of the displacement variables d, unit vectors , coordinate vertices , and the crossings (if applicable) as defined above. The departure point and are given by
e24
and
e25
respectively. If areas 1 and 2 are on a panel different than areas 3, 4, and 5, the formulas still apply; hence, there is no special treatment (if-else logic) for swept flux areas spanning panel edges. The iteration problem is well posed if the flow deformation prescribed by the SE fluxes satisfies the Lipschitz criterion. It is solved using the same Newton algorithm as described in section 3a. Note again that certain line integrals are static (e.g., the line integrals from the Eulerian vertices to the departure points) and do not need to be recomputed during each Newton iteration. Also, only at initialization it is necessary to compute intersections between line segments when and/or are nonempty. This is a major simplification compared to the “traditional” CSLAM algorithm.
Fig. 6.
Fig. 6.

A depiction of the categorization of possible flow cases defined in (15). The light blue shaded area is the first-guess swept flux. The dark blue filled circles are the departure points (that remain fixed) and the yellow filled circles are moved along the coordinate lines (according to red arrows) until the swept airmass flux matches the SE mass flux to round off. The middle and right columns showcase where one or two departure points coincide with a grid line.

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

The above algorithm constitutes a local iteration problem since a point along the side of the swept area is perturbed to obtain CSLAM fluxes consistent with the diagnosed SE fluxes. In other words, the vertex locations of the swept fluxes are determined by the orthogonal swept fluxes (illustrated in Figs. 3a–c). The upstream vertex locations stay fixed. We then introduce an extra degree of freedom along the flux side (which coincides with a coordinate line crossing else it is halfway in between the upstream vertices) and move that point (along the coordinate line) to make the SE and CSLAM fluxes match. Since the vertex locations stay fixed, the iteration only affects the swept flux area for the side in question. Therefore, the iteration algorithm is local and constrained on each side by the prescribed SE flux. As illustrated in Fig. 7, the union of the swept flux areas and the Eulerian cell define the equivalent upstream Lagrangian area. This equivalent upstream Lagrangian grid is shown in Fig. 7b for the new CSLAM algorithm. Note that had we chosen to perturb the departure points (Fig. 7a), then a global iteration problem would result since the departure points are shared between Lagrangian cells (or equivalently swept flux areas). This global iteration problem would also require intersections between Eulerian and Lagrangian cells to be computed during each iteration, which is an expensive operation. For example, if the global iteration problem would converge in 10 iterations, the cost (in the current implementation of the original CSLAM) would be equivalent to transporting approximately 90–140 tracers with the SE method (not shown).

Fig. 7.
Fig. 7.

A graphical illustration of two different ways to make CSLAM and SE consistent. The dark blue filled circles are the departure points, the dark blue lines are the sides of the Lagrangian CSLAM cells, and the black lines are the CSLAM control volumes. The arrows indicate the points that are perturbed to make SE and CSLAM consistent. (a) SE and CSLAM are made consistent by perturbing the departure points. Note that this problem is a global iteration problem requiring computation of overlap areas during each iteration. (b) The approach chosen in this paper. The intersections between the sides of the Lagrangian cells with the grid lines are perturbed. This constitutes a local iteration problem as fluxes are matched with SE fluxes for each flux side, whereas the departure points stay fixed.

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

4. Results

To assess the accuracy and efficiency of the CAM-SE–CSLAM algorithm, idealized baroclinic wave simulations have been performed with inert tracer distributions as well as idealized reactive chemical species. We compare the SE advection algorithm with the CSLAM advection algorithm. Note that the two transport schemes are driven by the same winds and airmass fields from the CAM-SE solution to the equations of motion. (The tests have been performed with the https://svn-homme-model.cgd.ucar.edu/branches/cslam@5168 code base.) This model version uses a floating Lagrangian vertical coordinate (Starr 1945; Lin 2004) but is otherwise equivalent to the CAM-SE version documented in Neale et al. (2012).

The model is run with 30 vertical levels (level locations as in CAM5), and the horizontal resolution is approximately 1° ( elements on each cubed-sphere panel and GLL quadrature points in each element). The time step for solving the equations of motion (including SE tracer transport) is 300 s, the hyperviscosity time step is 100 s, and the hyperviscosity coefficient is 1015 m4 s−1. High-resolution reference solutions are computed with a horizontal resolution of approximately ¼° ( elements on each cubed-sphere panel) with a dynamics time step of 75 s, a hyperviscosity time step of 12.5 s, and a hyperviscosity coefficient of 1013 m4 s−1. The vertical remapping is performed every third dynamics time step.

The CSLAM scheme has been implemented within the element structure (Erath et al. 2012) where each element has been subdivided into control volumes (using equiangular partitioning). Hence, the CSLAM grid resolution is also approximately 1° when using elements on each panel. Contrary to Lauritzen et al. (2010b), we use quadratic interpolation (instead of cubic) in the halo regions to interpolate cell average values on neighboring panels to halo cells resulting from extending the panel in question during the reconstruction process. CSLAM has been implemented for Courant numbers less than unity; hence, CSLAM is run with the remapping time step (i.e., 3 times longer time step than SE advection).

The idealized baroclinic wave used here is defined in Ullrich et al. (2014). This wave is similar to the Jablonowski and Williamson (2006) wave but with analytic support for deep atmosphere approximations. Both waves consist of a steady-state solution to the equations of motion with a perturbation that triggers the growth of a baroclinic wave. For this study, either of the baroclinic waves could have been used. In separate sections below, we describe the inert tracer transport test and reactive chemical species test, both being driven by the same SE solution to the Ullrich et al. (2014) baroclinic wave. For simplicity, we assume that the tracers are dimensionless.

a. Idealized baroclinic wave with inert tracers

Three (, , ) inert tracer distributions are used. The first tracer is a smooth “Gaussian ball” distribution:
e26
where η is the hybrid vertical coordinate,
e27
λ is longitude, θ is latitude, maximum amplitude in the vertical is placed at (which is near the surface where strong gradients occur during the baroclinic wave evolution), horizontal half-width (where R is the mean radius of Earth), and vertical half-width is 0.1. The great-circle distance r is measured as the distance between and the center of the initial distribution (, ) = (, ) = (40°N, 20°E):
e28
The initial condition is shown in row 1 in Fig. 8.
Fig. 8.
Fig. 8.

(columns from left to right) 1° SE, 1° CSLAM, and ¼° SE solutions for tracer one at (top) day 0, (middle) day 5, and (bottom) day 10 at level 20 (at approximately 610 hPa).

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

The second tracer distribution structurally resembles the initial condition temperature field, which is zonally symmetric (see Fig. 9):
e29
The third distribution is the slotted cylinder with discontinuous gradients (see Fig. 10):
e30
To assess the accuracy of CAM-SE using SE advection and the CSLAM transport scheme, we qualitatively and quantitatively compare the 1° solutions to the ¼° solutions. A standard normalized root-mean-square error norm is used to quantify the accuracy of the solution of ψ compared with a high-resolution reference solution :
e31
[e.g., appendix A in Lauritzen et al. (2012)] where I is the global integral. All data have been interpolated to a 1° regular latitude–longitude grid for plotting and analysis.
Fig. 9.
Fig. 9.

As in Fig. 8, but for tracer two and (top) day 0, (middle) day 8, and (bottom) day 10.

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

Fig. 10.
Fig. 10.

As in Fig. 8, but for tracer three . Note that the background value is nonzero and remains constant throughout the simulation; that is, the transport algorithms are consistent/free-stream preserving.

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

The wind and surface pressure evolution at these two resolutions are significantly different when the baroclinic wave starts growing and rapidly evolves with explosive cyclogenesis between model days 6 and 9. This is illustrated in the error norm for surface pressure shown in Fig. 11a, which is similar to the error norm analysis for the Jablonowski and Williamson (2006) wave. Since the winds and pressure fields driving the transport scheme differ significantly after day 9, the error norms for , , are dominated by wind and differences rather than differences in the accuracy of the transport algorithms. Up until days 6–9, however, the errors are dominated by the transport scheme errors if the tracer distributions are challenging enough for the transport scheme.

Fig. 11.
Fig. 11.

Standard error norm for (a) , (b) , (c) , and (d) using SE and CSLAM. The ¼° SE solution is used as the “truth” for computations.

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

Figure 11b shows as a function of time for computed with SE (blue line) and CSLAM (red line) at 1° resolutions, respectively, compared to the high-resolution reference solution computed with ¼° SE. Also, the black line is the error norm for the high-resolution solutions computed with CSLAM (ψ) and SE (). As expected, the high-resolution solutions differ a lot less (approximately an order of magnitude difference in ) than the low-resolution solutions (cf. red and blue lines with black lines). For the relatively smooth Gaussian initial condition , the CSLAM 1° solution is about twice as accurate compared to the SE solution until the baroclinic wave starts growing rapidly. This can also be seen in the instantaneous fields shown in Fig. 8.

For the zonally symmetric distribution (see Fig. 9), the errors for SE and CSLAM are completely dominated by the differences in the winds and pressure fields, which is expected for initially smooth and well-resolved tracer distributions (Fig. 11c). SE transport is formally fourth-order accurate (the addition of hyperviscosity, which scales as , implies that the scheme is 3.2-order accurate), and CSLAM is third order. For this smooth tracer, both transport schemes provide almost indistinguishable solutions consistent with both schemes being high order for smooth problems. Both schemes drop in formal accuracy (approximately second order; Lauritzen et al. 2014b) when a shape-preserving limiter is used.

In the case of the slotted-cylinder initial distribution (see Fig. 10), errors develop immediately because of the models inability to represent a distribution with discontinuous gradients. The CSLAM solution, however, produces errors that are approximately half that of SE until starts saturating by the driving flow differences. This is also visible in the instantaneous solutions shown in Fig. 10, where CSLAM resolves the sharp gradients more accurately than SE.

b. Idealized baroclinic wave with reactive chemical species

To assess the performance of SE and CSLAM in the presence of subgrid-scale forcing, we use a three-dimensional extension of the Lauritzen et al. (2015) idealized terminator test referred to as “the toy terminator chemistry test.” It consists of two reactive species and that initially add up to a constant
e32
and that react nonlinearly with each other. The chemical reactions are given in Lauritzen et al. (2015). We use the same initial conditions in each level for and as was used in Lauritzen et al. (2015). An advantage of this test is that no matter what flow the chemical species are placed in, the weighted sum (32) must always be conserved. So instead of using a prescribed analytical flow field as in Lauritzen et al. (2015), we use the idealized baroclinic wave flow field. The tendencies from the chemical reactions are computed every 900 s for the 1° computations. The tendencies are used to update the state every 900 s [the configuration is explained in appendix E in Lauritzen et al. (2015)].
As diagnostics, we used the averaged column integrated quantities:
e33
The initial condition for , , , and are shown in Fig. 12. The same fields at day 15 are shown in Fig. 13 for CSLAM and Fig. 14 for SE. Note that for SE and implied by CSLAM are identical. The SE transport scheme maintains a constant until day 11, after which errors start growing. In Lauritzen et al. (2015), this error growth was shown to be due to the monotone limiter from Guba et al. (2014). In Lopez and Taylor (2015), it was shown that this limiter preserves constant in exact arithmetic but not in the presence of round-off error. The CSLAM algorithm maintains the constant throughout the simulation.
Fig. 12.
Fig. 12.

Initial condition for (top left) , (top right) , (bottom left) , and (bottom right) used in the “toy” terminator chemistry test.

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

Fig. 13.
Fig. 13.

As in Fig. 12, but for day 15 using CSLAM.

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

Fig. 14.
Fig. 14.

As in Fig. 12, but for day 15 using SE.

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

c. Computational performance

Computational performance experiments are performed on the NCAR Yellowstone cluster (Computational and Information Systems Laboratory 2012) with standard computer optimization flags. We do not make use of threading. The amount of information that needs to be communicated between elements (if residing on different tasks) is very different between SE and CSLAM—similarly for the frequency of communication between elements. While the SE method only needs to share values at quadrature points located on the boundary of an element, communication is performed at every Runge–Kutta stage and every subcycling of the hyperviscosity operator. For the 1° model, that amounts to 21 communications per remapping time step (900 s), 5 communications per 300-s tracer time step, and 2 communications for hyperviscosity per tracer time step. CSLAM only needs one communication per 900-s tracer time step. That said, CSLAM needs all finite-volume values for each element surrounding the element being updated—that is, 72 values per tracer in the horizontal compared to 20 values for SE (four points on each edge plus the corner points). Figure 15a shows throughput for tracers for a 1-day simulation using SE (purple and black) and CSLAM (green) as a function of the number of tracers. The throughput for SE is shown for the CESM1 release code (purple line). The CAM-SE tracer advection code is currently being optimized further (J. Dennis 2016, personal communication). At a relatively low tracer count (at least for climate and climate-chemistry modeling), CSLAM is faster than SE. Note that the new CSLAM algorithm does not make use of an extensive search algorithm as the original formulation of CSLAM. The crossover between SE and CSLAM with the old formulation of flux-form CSLAM in HOMME (Harris et al. 2011) is over 20 tracers (not shown) and with the new consistent CSLAM algorithm, it is closer to 10 tracers. Figure 15a also shows the breakdown of computational cost for the different stages of the CSLAM algorithm. The departure point algorithm (orange line) and computation of high-order weights (yellow line) remains constant as a function of tracers and makes up a small cost even at low tracer counts. The communication (light blue), reconstruction (dark blue), and remapping (multiplication of reconstruction coefficients and precomputed weights; red) are all a function of tracers. The computation of the reconstruction coefficients is the most expensive step, mostly because of the reconstruction limiter that involves logic. About half of the cost of reconstruction is the boundary exchange (communication).

Fig. 15.
Fig. 15.

(a) Time (s) spent in various parts of the CSLAM transport algorithm for a 1-day simulation using 256 tasks as a function of number of tracers (the yellow line is on top of the orange line). The throughput for SE transport based on the CESM1 release (CAM-SE) is also shown. See text for details. (b) As in (a) but for a fixed number of tracers (40) and as a function of tasks. Note that for the rightmost data point (5400 tasks), there is only one element in the horizontal per processor. This is the limit of scalability for CAM-SE and CAM-SE–CSLAM without threading.

Citation: Monthly Weather Review 145, 3; 10.1175/MWR-D-16-0258.1

Especially for lower-resolution modeling, strong scaling is important. Figure 15b shows throughput as a function of tasks/processors. Note that the last data point on the right-hand side of the plot (5400 processors) is the throughput where there is only one element in the horizontal per processor. It is well known that SE exhibits strong scaling; however, CSLAM also scales to the limit of scalability despite the large halo.

Compared to the original formulation of CSLAM, the new CSLAM algorithm does not make use of an extensive search algorithm. It has been replaced with a local iterative algorithm to enforce consistency with SE. The communication structure (amount of data communicated and the frequency of communication), reconstruction, shape-preserving limiter, and remapping algorithm are essentially the same as the original CSLAM scheme. Hence, the detailed performance analysis as a function of cores and tracers presented in Erath et al. (2012) is also relevant for the algorithm presented here.

CAM-SE–CSLAM has been presented in the context of a dry dynamical core. The extension to a full physics setup requires several modifications to the existing modeling system. First of all, one need to make choices about which grid is used to run the physics parameterizations (quadrature grid or CSLAM control volumes). In either case, tendencies from the parameterizations must be mapped between grids. Second, the treatment of moisture in the dynamical core can complicate the coupling between CAM-SE and CSLAM. Currently, CAM-SE is based on a vertical coordinate based on moist (full) surface pressure, which implies that the model levels move when there are changes in the moisture field from parameterized sources/sinks of moisture. These sources/sinks need to be accounted for both in SE and CSLAM. To avoid this, dry-mass vertical coordinates, in which the model levels are determined by dry surface pressure (as well as the usual hybrid coefficients), can be used. With dry-mass vertical coordinates, the model levels do not change because of parameterized water vapor sources/sinks. The infrastructure to run physics, tracers, and dynamics on different grids has been added to CAM, and a dry-mass vertical coordinate version of CAM-SE has been derived and implemented. This version also easily accounts for condensate loading. Details on these developments are beyond the scope of this paper.

5. Summary

In this paper, it has been shown that it is possible to consistently couple CAM-SE with the CSLAM transport algorithm without violating desirable properties such as shape preservation, mass conservation, linear correlation preservation, and consistency (aka compatibility, free-stream preservation). This is achieved by first deriving formulas for diagnosing SE airmass flux through the CSLAM control volume faces. These mass fluxes are used to construct CSLAM swept areas for which the airmass fluxes computed with CSLAM match the SE airmass fluxes to round off. The iterative algorithm is local and results in an equivalent upstream Lagrangian CSLAM grid that spans the sphere without gaps or overlaps. This new consistent SE-CSLAM algorithm avoids the expensive search for overlap areas used in the original formulations of CSLAM. The source code for computing weights has almost been cut in half with the new algorithm compared to the original CSLAM. This is mainly due to the new algorithm being formulated more generally, so special case statements for the edges and corners of the sphere have been reduced to a minimum.

It has been shown in idealized tracer experiments that CSLAM is more accurate than SE for challenging tracer distributions and at the same time is more efficient that SE if one is using more than approximately 12–15 tracers. Also, the CAM-SE–CSLAM model retains the excellent strong scaling properties of CAM-SE. The algorithm for making the CSLAM Lagrangian grid consistent with SE is general in the sense that CSLAM can be made consistent with any airmass fluxes as long as the fluxes satisfy the Lipshitz criterion.

Acknowledgments

Steve Goldhaber, James Overfelt, Paul Ullrich, and Mark Taylor were supported by the U.S. Department of Energy Office of Biological and Environmental Research, Work Package 12-015334 “Multiscale Methods for Accurate, Efficient, and Scale-Aware Models of the Earth System.”

APPENDIX

Spectral Element Subcell Flux

Our goal is to write each of the three terms on the right-hand side of (8) as a sum of edge fluxes in such a way that the fluxes along an edge shared by two subcells will be equal in magnitude and of opposite sign.

a. Strong form divergence:

The divergence in curvilinear coordinates is given by
ea1
where J is the Jacobian and are the contravariant components of the flux vector .
To compute the implied spectral element flux over the th CSLAM control volume, we construct special test functions such that
ea2
for all polynomials of degree . Such test functions must exist since the integral is a linear functional of p, and p is uniquely determined by its GLL nodal values. One can further show that is a tensor product of one-dimensional test functions . Hence, the flux for the th CSLAM control volume is
ea3
Thus, the flux naturally decomposes into the integral of along each edge. These can be computed numerically based on the GLL nodal values via
ea4
where is a polynomial interpolation matrix defined so that
ea5
Each term on the right-hand side of (A4) constitutes the contribution to from the strong flux divergence.

We note that in the SE method, at an edge shared by neighboring elements, both J, , and will not necessarily be continuous because of deformed elements and different contravariant coordinates used within each element. However, the flux is continuous at element boundaries if is continuous (Taylor and Fournier 2010).

b. Weak form diffusion:

This term, where , does not change the mass within an element and thus has zero net flux at element boundaries. It does induce mixing between subcells and will change the nodal values of . The DSS operation can then redistribute some of this mass into neighboring cells. However, that flux is accounted for in the DSS operation below, not here.

For each subinterval, we need to decompose the change in mass into left, right, top, and bottom fluxes: , , , and , respectively. The requirement, that the flux at the sides of the element is zero and that the fluxes at CSLAM control volume edges are equal and with opposite signs, is sufficient to uniquely determine all the fluxes.

In each coordinate direction, we have
ea6
(similarly for the top and bottom). Starting with the leftmost () subinterval, we know that , and then (A7) can be used to determine . Since the fluxes at adjacent subelement boundaries must be equal and opposite, this determines , and as before, once is determined, (A6) determines .

Since is a partition of unity, it can be shown that this procedure will end with the flux at the interval endpoint of , as required.

Thus these fluxes are given by
ea7
ea8
In two dimensions, we first integrate by parts (weak form):
ea9
Then using the results from above, we get
ea10
ea11
and we can write the diffusion term as a sum of four edge fluxes:
ea12
Each term on the right-hand side of (A12) constitutes the contribution to from the diffusion operator.

c. DSS operation:

We now treat the edge flux term coming from the DSS operation. First, consider the DSS term along a coordinate line: In one dimension, the net change in mass in the entire element is given by
ea13
where we have used the fact that at element interior points. This term naturally decomposes into a flux at the left and right interval endpoints
ea14
ea15
with
ea16
We first note that this flux is conservative at the element level since the flux on the right edge of one element will be equal and opposite to the flux on the left edge of its adjacent element.
As for the diffusion term, given , the remaining fluxes are determined by requiring that adjacent fluxes are equal and opposite, , and that the change in mass from the DSS term in each subinterval is given by the sum of the subinterval fluxes:
ea17
The fact that is a partition of unity ensures that the final flux computed by this procedure, , will satisfy (A15).
The result can be written as
ea18
ea19
In two dimensions, we have
ea20
Applying the one-dimensional (1D) procedure from above to this term and denoting the resulting fluxes (which are now a function of ) at the top and bottom edges by and , respectively, we get
ea21
ea22
so that
ea23
Each term on the right-hand side of (A23) constitute the contribution to from the DSS operator.

The definition of and is a simple extension of and as defined in (A18) and (A19). The only difficulty is at the corners, , where some ambiguity exists in how D is conservatively distributed among the element edges that share the corner node—that is, . We use the fact that all the DSS terms from all the elements that share a corner must sum to zero, , where the sum is taken over the three or four elements, e, that share a single corner point, . We can then divide the contributions among each of the three or four elements, noting that even elements that share only a single node and not an edge will have flux contributions that flow through intermediate elements. For three elements sharing a point, the flux is based on one-third the difference of the DSS terms across neighboring faces. The formula for four elements is similar.

REFERENCES

  • Barth, T. J., and D. C. Jespersen, 1989: The design and application of upwind schemes on unstructured meshes. Proc. AIAA 27th Aerospace Sciences Meeting, Reno, NV, American Institute of Aeronautics and Astronautics, 112, doi:10.2514/6.1989-366.

    • Crossref
    • Export Citation
  • Computational and Information Systems Laboratory, 2012: Yellowstone: IBM iDataPlex system. NCAR/UCAR. [Available online at http://n2t.net/ark:/85065/d7wd3xhc.]

  • Dawson, C., 1999: Conservative, shock-capturing transport methods with nonconservative velocity approximations. Comput. Geosci., 3, 205227, doi:10.1023/A:1011539311766.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deleersnijder, E., and G. Lebon, 2001: Enforcing the continuity equation in numerical models of geophysical fluid flows. Appl. Math. Lett., 14, 867873, doi:10.1016/S0893-9659(01)00057-X.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dennis, J., A. Fournier, W. Spotz, A. St-Cyr, M. A. Taylor, S. J. Thomas, and H. Tufo, 2005: High-resolution mesh convergence properties and parallel efficiency of a spectral element atmospheric dynamical core. Int. J. High Perform. Comput. Appl., 19, 225235, doi:10.1177/1094342005056108.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Dennis, J., and Coauthors, 2012: CAM-SE: A scalable spectral element dynamical core for the Community Atmosphere Model. Int. J. High. Perform. Comput. Appl., 26, 7489, doi:10.1177/1094342011428142.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deville, M. O., P. F. Fischer, and E. H. Mund, 2002: High-Order Methods for Incompressible Fluid Flow. Cambridge Monographs on Applied and Computational Mathematics, No. 9, Cambridge University Press, 528 pp.

    • Crossref
    • Export Citation
  • Erath, C., and R. D. Nair, 2014: A conservative multi-tracer transport scheme for spectral-element spherical grids. J. Comput. Phys., 256, 118134, doi:10.1016/j.jcp.2013.08.050.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Erath, C., P. H. Lauritzen, J. H. Garcia, and H. M. Tufo, 2012: Integrating a scalable and efficient semi-Lagrangian multi-tracer transport scheme in HOMME. Procedia Comput. Sci., 9, 9941003, doi:10.1016/j.procs.2012.04.106.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Erath, C., P. H. Lauritzen, and H. M. Tufo, 2013: On mass conservation in high-order high-resolution rigorous remapping schemes on the sphere. Mon. Wea. Rev., 141, 21282133, doi:10.1175/MWR-D-13-00002.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gross, E. S., L. Bonaventura, and G. Rosatti, 2002: Consistency with continuity in conservative advection schemes for free-surface models. Int. J. Numer. Methods Fluids, 38, 307327, doi:10.1002/fld.222.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Guba, O., M. Taylor, and A. St-Cyr, 2014: Optimization-based limiters for the spectral element method. J. Comput. Phys., 267, 176195, doi:10.1016/j.jcp.2014.02.029.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hall, D. M., P. A. Ullrich, K. A. Reed, C. Jablonowski, R. D. Nair, and H. M. Tufo, 2016: Dynamical Core Model Intercomparison Project (DCMIP) tracer transport test results for CAM-SE. Quart. J. Roy. Meteor. Soc., 142, 16721684, doi:10.1002/qj.2761.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Harris, L. M., P. H. Lauritzen, and R. Mittal, 2011: A flux-form version of the Conservative Semi-Lagrangian Multi-tracer transport scheme (CSLAM) on the cubed sphere grid. J. Comput. Phys., 230, 12151237, doi:10.1016/j.jcp.2010.11.001.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Jablonowski, C., and D. L. Williamson, 2006: A baroclinic instability test case for atmospheric model dynamical cores. Quart. J. Roy. Meteor. Soc., 132, 29432975, doi:10.1256/qj.06.12.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kuo, H.-C., and R. T. Williams, 1990: Semi-Lagrangian solutions to the inviscid Burgers equation. Mon. Wea. Rev., 118, 12781288, doi:10.1175/1520-0493(1990)118<1278:SLSTTI>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., and J. Thuburn, 2012: Evaluating advection/transport schemes using interrelated tracers, scatter plots and numerical mixing diagnostics. Quart. J. Roy. Meteor. Soc., 138, 906918, doi:10.1002/qj.986.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., C. Jablonowski, M. A. Taylor, and R. D. Nair, 2010a: Rotated versions of the Jablonowski steady-state and baroclinic wave test cases: A dynamical core intercomparison. J. Adv. Model. Earth Syst., 2, 15, doi:10.3894/JAMES.2010.2.15.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., R. D. Nair, and P. A. Ullrich, 2010b: A Conservative Semi-Lagrangian Multi-tracer transport scheme (CSLAM) on the cubed-sphere grid. J. Comput. Phys., 229, 14011424, doi:10.1016/j.jcp.2009.10.036.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., P. A. Ullrich, and R. D. Nair, 2011: Atmospheric transport schemes: Desirable properties and a semi-Lagrangian view on finite-volume discretizations. Numerical Techniques for Global Atmospheric Models, P. H. Lauritzen et al., Eds., Lecture Notes in Computational Science and Engineering, Vol. 80, Springer, 185–250, doi:10.1007/978-3-642-11640-7_8.

    • Crossref
    • Export Citation
  • Lauritzen, P. H., W. C. Skamarock, M. J. Prather, and M. A. Taylor, 2012: A standard test case suite for two-dimensional linear transport on the sphere. Geosci. Model Dev., 5, 887901, doi:10.5194/gmd-5-887-2012.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., J. T. Bacmeister, T. Dubos, S. Lebonnois, and M. A. Taylor, 2014a: Held–Suarez simulations with the Community Atmosphere Model spectral element (CAM-SE) dynamical core: A global axial angular momentum analysis using Eulerian and floating Lagrangian vertical coordinates. J. Adv. Model. Earth Syst., 6, 129140, doi:10.1002/2013MS000268.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., and Coauthors, 2014b: A standard test case suite for two-dimensional linear transport on the sphere: Results from a collection of state-of-the-art schemes. Geosci. Model Dev., 7, 105145, doi:10.5194/gmd-7-105-2014.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lauritzen, P. H., A. J. Conley, J.-F. Lamarque, F. Vitt, and M. A. Taylor, 2015: The terminator “toy” chemistry test: A simple tool to assess errors in transport schemes. Geosci. Model Dev., 8, 12991313, doi:10.5194/gmd-8-1299-2015.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lin, S.-J., 2004: A “vertically Lagrangian” finite-volume dynamical core for global models. Mon. Wea. Rev., 132, 22932307, doi:10.1175/1520-0493(2004)132<2293:AVLFDC>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lin, S.-J., and R. B. Rood, 1996: Multidimensional flux-form semi-Lagrangian transport schemes. Mon. Wea. Rev., 124, 20462070, doi:10.1175/1520-0493(1996)124<2046:MFFSLT>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lopez, N. A., and M. A. Taylor, 2015: Improving the tracer correlation problem in a spectral element dynamical core. Center for Computing Research Summer Proc. 2015, Albuquerque, NM, Sandia National Laboratories, 106–117. [Available online at https://cfwebprod.sandia.gov/cfdocs/CompResearch/docs/proceedings/ccr15.pdf#page=112.]

  • Nair, R. D., and P. H. Lauritzen, 2010: A class of deformational flow test cases for linear transport problems on the sphere. J. Comput. Phys., 229, 88688887, doi:10.1016/j.jcp.2010.08.014.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nair, R. D., J. S. Scroggs, and F. H. M. Semazzi, 2002: Efficient conservative global transport schemes for climate and atmospheric chemistry models. Mon. Wea. Rev., 130, 20592073, doi:10.1175/1520-0493(2002)130<2059:ECGTSF>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Nair, R. D., H.-W. Choi, and H. M. Tufo, 2009: Computational aspects of a scalable high-order discontinuous Galerkin atmospheric dynamical core. Comput. Fluids, 38, 309319, doi:10.1016/j.compfluid.2008.04.006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Neale, R. B., and Coauthors, 2012: Description of the NCAR Community Atmosphere Model (CAM 5.0). NCAR Tech. Note NCAR/TN-486+STR, 274 pp. [Available online at http://www.cesm.ucar.edu/models/cesm1.0/cam/docs/description/cam5_desc.pdf.]

  • Pudykiewicz, J., R. Benoit, and A. Staniforth, 1985: Preliminary results from a partial LRTAP model based on an existing meteorological forecast model. Atmos.–Ocean, 23, 267303, doi:10.1080/07055900.1985.9649229.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Putman, W. M., and S.-J. Lin, 2007: Finite-volume transport on various cubed-sphere grids. J. Comput. Phys., 227, 5578, doi:10.1016/j.jcp.2007.07.022.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rančić, M., R. J. Purser, and F. Mesinger, 1996: A global shallow-water model using an expanded spherical cube: Gnomonic versus conformal coordinates. Quart. J. Roy. Meteor. Soc., 122, 959982, doi:10.1002/qj.49712253209.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rood, R. B., 1987: Numerical advection algorithms and their role in atmospheric transport and chemistry models. Rev. Geophys., 25, 71100, doi:10.1029/RG025i001p00071.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Shashkin, V. V., and M. A. Tolstykh, 2016: Parallel implementation of the cascade mass-conserving semi-Lagrangian transport scheme. Russ. J. Numer. Anal. Math. Modell., 31, 1728, doi:10.1515/rnam-2016-0002.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Starr, V. P., 1945: A quasi-Lagrangian system of hydrodynamical equations. J. Meteor., 2, 227237, doi:10.1175/1520-0469(1945)002<0227:AQLSOH>2.0.CO;2.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • St-Cyr, A., C. Jablonowski, J. M. Dennis, H. M. Tufo, and S. J. Thomas, 2008: A comparison of two shallow-water models with nonconforming adaptive grids. Mon. Wea. Rev., 136, 18981922, doi:10.1175/2007MWR2108.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Taylor, M. A., 2011: Conservation of mass and energy for the moist atmospheric primitive equations on unstructured grids. Numerical Techniques for Global Atmospheric Models, P. H. Lauritzen et al. Eds., Lecture Notes in Computational Science and Engineering, Vol. 80, Springer, 357–380, doi:10.1007/978-3-642-11640-7_12.

    • Crossref
    • Export Citation
  • Taylor, M. A., and A. Fournier, 2010: A compatible and conservative spectral element method on unstructured grids. J. Comput. Phys., 229, 58795895, doi:10.1016/j.jcp.2010.04.008.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Taylor, M. A., J. Tribbia, and M. Iskandarani, 1997: The spectral element method for the shallow water equations on the sphere. J. Comput. Phys., 130, 92108, doi:10.1006/jcph.1996.5554.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ullrich, P. A., P. H. Lauritzen, and C. Jablonowski, 2009: Geometrically exact conservative remapping (GECoRe): Regular latitude–longitude and cubed-sphere grids. Mon. Wea. Rev., 137, 17211741, doi:10.1175/2008MWR2817.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Ullrich, P. A., T. Melvin, C. Jablonowski, and A. Staniforth, 2014: A proposed baroclinic wave test case for deep- and shallow-atmosphere dynamical cores. Quart. J. Roy. Meteor. Soc., 140, 15901602, doi:10.1002/qj.2241.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zarzycki, C. M., M. N. Levy, C. Jablonowski, J. R. Overfelt, M. A. Taylor, and P. A. Ullrich, 2014: Aquaplanet experiments using CAM’s variable-resolution dynamical core. J. Climate, 27, 54815503, doi:10.1175/JCLI-D-14-00004.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Zerroukat, M., N. Wood, and A. Staniforth, 2002: SLICE: A semi-Lagrangian inherently conserving and efficient scheme for transport problems. Quart. J. Roy. Meteor. Soc., 128, 28012820, doi:10.1256/qj.02.69.

    • Crossref
    • Search Google Scholar
    • Export Citation
Save