MAY 1986 FULTON, ClES1ELSKI AND SCHUBERT 943REVIEWMultigrid Methods for Elliptic Problems: A Review- SCOTT R. FULTON, PAUL E. CIESIELSKI AND WAYNE H. SCHUBERTDepartment of Atmospheric Science, Colorado State University, Fort Collins, CO 80523(Manuscript received 14 May 1985, in final form 1 July 1985)ABSTRACT Multigrid methods solve a large class of problems very efficiently. They work by approximating a problemon multiple overlapping grids with widely varying mesh sizes and cycling between these approximations, usingrelaxation to reduce the error on the scale of each grid. Problems solved by multigrid methods include generalelliptic partial differential equations, nonlinear and eigenvalue problems, and systems of equations from fluiddynamics. The efficiency is optimal: the computational work is proportional to the number of unknowns. This paper reviews the basic concepts and techniques of multigrid methods, concentrating on their role asfast solvers for elliptic boundary-value problems. Analysis of simple relaxation schemes for the Poisson problemshows that their slow convergence is due to smooth error components; approximating these components on acoarser grid leads to a simple multigrid Poisson solver. We review the principal elements of multigrid methodsfor more general problems, including relaxation schemes, grids, grid transfers, and control algorithms, plustechniques for nonlinear problems and boundary conditions. Multigrid applications, current research, andavailable software are also discussed. CONTENTS1. Introduction2. Classical methods a. Gaussian elimination b. Point iterative methods c. ADI method d. Discussion3. Basic multigrid concepts a. Analysis of relaxation b. A two-grid method c. A multigrid method4. Survey ofmultigrid techniques a. Relaxation schemes b. Grids c. Grid transfers d. Control algorithms 1) Cycling algorithms 2) Full Multigrid (FMG) algorithms e. The Full Approximation Scheme (FAS) f. Boundary conditions g. Generalizations5. Applications6. Concluding remarks1. Introduction Elliptic boundary-value problems arise in manyareas of geophysical fluid dynamics. Perhaps the simplest type is the Poisson problem -V2u =f, (l.1)which occurs in a variety of contexts. For example, innondivergent barotropic models 72 is the two dimensional horizontal Laplacian, u the unknown streamfunction, f the known relative vorticity, and (1.1) issolved with Dirichlet boundary conditions. In Boussinesq models of shallow convection V2 is the threedimensional Laplacian, u the unknown nonhydrostaticpressure, and fa function of the known temperatureand motion fields, and (1.1) is solved with Neumannboundary conditions (e.g., Ogura and Phillips, 1962).Another simple elliptic problem which often arises isthe Helmholtz problem. An example is in the application of semi-implicit time differencing to primitiveequation models which use finite differences in space(e.g., Haltiner and Williams, 1980, p. 149). Then urepresents the geopotential at the present time leveland fis known in terms of the geopotential and windfields at previous time levels. Not all the elliptic boundary-value problems whicharise in geophysical fluid dynamics have constant coefficients. As a prototype of variable coefficient problemsin two dimensions let us consider (aUx)x + (buy)y -- f (1.2)which can be classified as a problem with weakly orstrongly varying coe~cients depending on the percentage variation of the known coefficients a(x, y) andb(x, y). In the balanced vortex theory of hurricane development (e.g., Schubert and Hack, 1983) we encounter an equation of the form (1.2) with u representing the streamfunction for the transverse part ofthe mass cimulation; the coefficients involve the poc 1986 American Meteorological Society944MONTHLY WEATHER REVIEWVOLUME 1 14 tential vorticity and the inertial stability and become strongly varying as the vortex develops. Variable coef ficient problems in three dimensions arise in both quasi-geostrophic and semi-geostrophic theory (e.g., Hoskins, 1975; Hoskins and Draghici, 1977). Examples include the omega equations, tendency equations, and the relations between geopotential and potential vor ticity. Occasionally, nonlinear boundary-value problemsare also encountered. For example, in the balancedvortex theory of hurricanes there arises a two-dimensional nonlinear problem relating the unknown potential function to the known Ertel potential vorticity(Thorpe, 1985). In more general three-dimensionalnonlinear balance models and in certain initializationschemes (e.g., Haltiner and Williams, 1980, pp. 215,368) one must solve a nonlinear problem which relatesthe streamfunction for the rotational flow to the knowngeopotential field. If the elliptic boundary-value problem needs to besolved many times (e.g., at every time step in evolutionproblems), an efficient solution procedure is important,especially in three-dimensional problems. The subjectof this review paper is the multigrid method, which canbe regarded as an approach to developing fast solversfor all of the above examples of elliptic boundary-valueproblems in geophysical fluid dynamics. Solving an elliptic problem is usually a two-stageprocedure. First, one approximates the continuousproblem by a discrete analogue, usually based on finitedifference approximations over a sufficiently fine grid.Second, one solves the resulting discrete equations,usually by an iterative technique. For the simplestproblems, such as the Poisson problem, there exist "fastsolvers" which give the solution with optimal efficiency,i.e., computational work proportional to the numberof unknowns. However, for more complicated problems iterative schemes tend to be slow, and typically'get even slower as the resolution of the grid increases. In the early 1970s, multigrid methods were introduced as a means of constructing fast solvers for generalelliptic problems. While various components of multigrid processes had been known earlier, especially fromthe work of Fedorenko (1962, 1964), Achi Brandt(1973) was the first to demonstrate the practical efficiency and generality of multigrid methods. The fundamental components were also developed independently by W. Hackbusch and by P. Frederickson atabout the same time. Multigrid methods have beenanalyzed, applied and generalized in many ways sincetheir introduction, and are gradually becoming recognized as a powerful tool in applied mathematics. Multigrid methods differ from the perhaps more familiar nested grid techniques in that the primary useof multiple grids is not to obtain nonuniform resolution. Instead, the basic multigrid idea is to approximatethe same continuous problem on a set of overlappinguniform grids of widely varying mesh sizes, and cyclebetween these discrete problems to produce the solutionon the finest grid with optimal efficiency. Each grid isemployed specifically in solving for those componentsof the solution which require the corresponding resolution; error in these components is reduced efficientlyusing a simple relaxation scheme. The computationalcost of the coarser grids is small since they involverelatively few points. The primary advantages of multigrid methods areefficiency and generality. Defining a work unit as thenumber of operations needed simply to express thediscrete equations, the typical multigrid efficiency is tosolve a problem to the level of truncation error in justa few (4 to 10) work units. This efficiency has beenobtained for a wide class of problems, including generalelliptic boundary-value problems, singular perturbationand nonelliptic problems, minimization problems, andintegral equations. Self-adaptive local mesh refinementcan be incorporated in a natural way, and nonlinearproblems are solved with the same efficiency as linearproblems. This paper gives a concise review of the basic concepts and techniques of multigrid methods. It is intended both to serve as a simple introduction to thesemethods and to call attention to their potential forsolving problems in geophysical fluid dynamics. Theemphasis here is on methods for solving a single ellipticequation such as those discussed above; more detail,plus information on methods for more complicatedproblems, is available from a number of sources. Thefirst comprehensive treatment of multigrid methodswas Brandt (1977a), which remains very useful. Thereview article of Stiiben and Trottenberg (1982) contains detailed analyses of simple model problems, alongwith some interesting historical notes. Both of the abovesources also contain listings of sample multigfid programs. Brandt (1982) is an invaluable practical guideto developing multigrid algorithms for a wide range ofproblems; an updated version (Brandt, 1984)containsadditional introductory material plus detailed information on multigrid methods for systems of equationsin fluid dynamics. Comprehensive multigrid bibliographies are given in all of these sources and in Brand(1982). In the body of this paper we first review several classical methods for solving discretized elliptic boundaryvalue problems, including Gaussian elimination, pointiterative methods, and an ADI method. These are applied to the Poisson problem in section 2 to illustratethe amount of computational work they require. Thereason for the slow convergence of relaxation methodsis analyzed in section 3, leading naturally to the basicmultigrid concept of using corrections computed oncoarser grids. A simple two-grid method is describedand then generalized to give a workable multigfidmethod. Developing a multigrid algorithm for a particularproblem involves choosing a relaxation scheme, a setMAY 1986 FULTON, CIESIELSKI AND SCHUBERT 945of grids, grid transfer operators, and a control algorithm. In section 4 we examine each of these principalcomponents in more detail. Also discussed are the FullApproxmation Scheme for nonlinear problems andadvanced techniques, basic principles governing thetreatment of boundary conditions, and some generalizations to finite-element and spectral discretizationsand other types of problems. Section 5 surveys recentapplications of multigrid methods, areas of current research, and available software. Concluding remarks aregiven in section 6.2. Classical methods As a prototype elliptic problem, consider the Poissonproblem Lu -- ~~2[/=f in fi (2.1) u = g on O~2where X72 = 032/~0c2 -~ 02/~y2 is the Laplacian operator,~2 = [0, 1] x [0, 1] is the unit square with boundary0fi, andfand g are regarded as known. Equation (2.1)may be approximated on the gridflh = (Xj, Yk) = (jh, kh): 0 ~< j, k < N, h = l/N (2.2)using second-order centered finite differences ash-2[4Ujk- Uj-i,k- U~+l,k- Uj,k-~ -- U~.k+~] =f~k (0<j,k<N), (2.3a) Ujk=gjk (J=0, j=N,k=0, ork=N). (2.3b)Here ujk is the discrete approximation to the true solution u(x, y) at the point (xj, Yk), JSk = f(xj, Yk) and g~k= g(x2, Yk). Let us denote the discrete problem (2.3a)by the shorthand notation Lhuh = fh, (2.4)where Lh is a linear operator (the discrete approximation to L = -T2) and un and f* are "grid functions"consisting of the values ujk andJ~k on the grid fth. Next,we consider several classical methods for solving thisproblem.a. Gaussian elimination Eliminating the boundary values using (2.3b), thediscrete problem (2.4) may be expressed in matrix formas Au = f. (2.5)Here u and f represent the grid functions uh and f*with values taken in lexicographic order, e.g., u = (u~ ~,/~/21, - - - , ~/N-I,I, ~t12, ~t22, - - - , /~N-I,2, - - - , UN-I,N-I)Twith T denoting the transpose. The matrix A has dimension (N- 1)2 by (N- 1)2 with the structure shownin Fig. 1, which can be viewed as either banded orblock-tridiagonal. It is nonsingular and hence the discrete problem (2.4) is uniquely solvable. Standard4 -, r_i- ..... -I-I 4-1 -I i !~1 4-I -I I-I 4-1 -I ~-I 4 -I I I.........-I 14 -I I-I-I LI 4-I ~l -I 4-1 -I I -I 4-1 -II -I 4 I -I I L I II -I I -I I ~11 II-I 4-I I -I III -I 4-I I -I II -I 4 -I I -I II -I 41 -II'._~ ..... r4_~ ~.lII -I I-I 4-1 -I ii -I -I 44 I~ -I,, -, -, ., -I 41 -I .... I -I i-I 4.-I I -I I -I 4-I I - I II -I 4 -I I t -I 1 -I 4FIG. 1. Structure of the matrix A in (2.5), shown for N = 6 after multiplying by h2.Gaussian elimination can be used to solve (2.5); thisis a direct method in that (with exact arithmetic) ityields the exact (discrete) solution in a finite numberof operations. Treating A as a band matrix of bandwidth 2N-l, this method requires O(Nn) operations[O(N4) for the decomposition and O(N3) for the forward and backward substitution] and O(Ns) storage.However, the true sparsity of A (the fact that A hasonly five nonzero bands) is not utilized. Gaussianelimination based on the block-tridiagonal point ofview, e.g., the algorithm of Lindzen and Kuo (1969),requires the same number of operations and storage,i.e., O(N4) and O(NS), respectively.b. Point iterative methods In contrast to direct methods, indirect or iterativemethods do not generate the true (discrete) solutionitself, but rather a sequence of approximations whichconverge to it. In this paper we will consistently usea tilde to denote such approximations. In a point iterative method the values ~jk of an approximation ~hare modified point by point to obtain a new approximation ~new. For example, the Jacobi method (alsoknown as simultaneous relaxation) defines the newvalue fi)~ at a point (xi, Yk) by requiring that the discrete equation (2.3a) hold at that point using the oldvalues at the surrounding points, so thath-214a~/w - aj-l,k- aj+l.k- aj,k-I -- aj,k+l] ~ (2.6)A Jacobi "sweep" consists of computing ~?~w at all(interior) points via (2.6) and then replacing the oldapproximation ffh by ffh~,~. This method requires O(N2)operations per sweep and O(N2) storage.946 MONTHLY WEATHER REVIEW VOLUME 114 The Gauss-Seidel method (also known as successiverelaxation) is similar to the Jacobi method except thateach value is updated immediately after it is calculated.In contrast to the Jacobi method, the order in whichthe points are taken is significant. Using lexicographicordering [i.e., taking the points (xj, Yk) in the order(j, k) = (l, l), (2, l) ..... (N- l, 1), (2, l) .... ,(N- I, N - 1)] the Gauss-Seidel relaxation equationish-:[4~/~ - t~j'_*~,k - u~j~'5'-, - t~,+,l = Jj~. (2.7)This equation gives t~?~w 'explicitly since t~?_'Y.a andu'?,[5~ have been computed before reaching the point(x-, Y0. As before, O(N2) operations per sweep-and0(N2) storage are required; however, the storage is actually less than for the Jacobi method since the newvalues may be written 6vet the old values immediately. The Gauss-Seidel method can be viewed as addinga correction fi?~w _ fij~, to each value t~-~ successively.The rate of convergence may be accelerated by scalingthis correction by a relaxation parameter o~ before adding it, leading to the SOR (successive over-relaxation)method defined by 0.) ~llew + aj+~,k + a~,?,~_, + a-,~+,l. (2.8)Again, O(N2) operations per sweep and O(N2) storageare required, with the SOR method reducing to theGauss-Seidel method if w -- 1. The convergence of the above methods can be described by writing them as fixed-point iterations so thatthe errors vn = un - fin satisfy v~n~ = Tnv~, (2.9)where Ta is an iteration operator. From (2.9) the iteration converges (for any initial approximation) if andonly if the spectral radius ~a(Ta) is less than one, withp(Th) providing an estimate of the factor by which theerror will be reduced by one relaxation sweep. For thediscrete Poisson problem cons~dered above it can beshown (e.g., Stoer and Bulirsch, 1980) that the optimalrate for the SOR method is obtained with the relaxationparameter w = 2/[ 1 + sin0rh)], and that cos(;rh) = I 5 + O(h4)' Jacobi cos:(a'h) = 1 - r2h2 + O(ha),~(Ta) = Gauss-Seidel (2.10) I - sin(rh) = I - 2~rh + O(h2), I + sin0rh)SOR (optimal).Thus the rate of convergence of each of these methodsdepends on the resolution, with higher resolution(smaller h) resulting in slower convergence. Since eachsweep essentially reduces the error by the factor p(Th),reducing the error by a specified factor e requires ssweeps, where [p(Tn)]s = e. Solving for s and using(2.10) gives - p/:~r:h2 + O(h~)]-~ = O(N~), Jacobis (-ln0x [;r:h2 + O(h4)]-t = O(N2)' Gauss-Seidel [2~rh + O(h2)]-~ = O(N), SOR (optimal).(2. It)Since each method requires O(N:) operations persweep, the total work required to reduce the error bya specified factor ~ is O(Nn) operations for the Jacobiand Gauss-Seidel methods and O(Na) operations forthe optimal SOR method.c. ~tDI method In contrast to the point iterative methods describedabove, block iterative methods produce a new approximation ~an,w from a previous one ~n by simultaneouslymodifying the values of a "block" or set of points. Asan example, consider the following ADI (alternatingdirection implicit) method, in which the "blocks" arelines of points in the x and y dir6ctions. Each ADIsweep consists of two steps, each based on the discreteEq. (2.3a) in the form(-u~-~,~ + 2u~ - u~+~,0 + (-uj,~-t + 2uj~ - u~.k+~) = h2f~. (2.12)The first step treats the x-dependence implicitly to produce an intermediate approximation t~in, t via-a3n\,~ + (2 + r)a3P,' =. h2f~ - [-a;,&_~ + (2 - r)a~- aj,k+d, (2.13a)where r is an iteration parameter, and the second steptreats the y-dependence implicitly to obtain the newapproximation/2~w via-a~,~_, + (2 + r)a~,t'~ = h~j~ - [fi)~-h,~ + (2 - r)fi)~,t - fi)~,~]. (2.13b)Each step involves solving N tridiagonal linear systemsof order N; thus the ADI method requires O(N2) operations per sweep and O(N2) storage. For the Poisson problem the above ADI method canbe shown to converge if a fixed positive iteration parameter r is used for all sweeps. The optimal constantr is 2 sin0rh), for which the method converges at theM^Y1986 FULTON, CIESIELSKI AND SCHUBERT 947same rate as the optimal SOR method. Faster convergence can be obtained by allowing r to vary from sweepto sweep. A simple algorithm for determining the optimal set of n parameters, with n a power of 2, is givenin Stoer and Bulirsch (1980). One cycle ofn ADI sweepsusing these parameters reduces the error by a factor ofapproximately 1 - 8(~h)TM, SO that n s ~ - ~ (-~rh)-l/n In e ADIsweeps are required to reduce the error by a specifiedfactor e. This number is minimized by choosing n ~.ln(4Drh), yielding s = O(lnN). Since O(N2) operationsare required per sweep, the ADI method requires O(N2InN) operations to reduce the error by a specified factor. d. Discussion With low resolution (small N), direct solution ofdiscretized elliptic problems by Gaussian elimination maybe practical. With higher resolution (larger N), iterativemethods become more attractive, in part because theyrequire less storage [O(N2) vs O(N4)] and in part because they are less sensitive to round-off error (inGaussian elimination round-off error tends to accumulate-albeit slowly--while in iterative methods itis automatically reduced by the iteration). The simplestiterative schemes (e.g., Jacobi and Gauss-Seidel relaxation) require O(N4) operations, as does Gaussianelimination. The SOR method reduces this operationcount to O(N3) by introducing a relaxation parameter;the ADI method lowers it to O(N2 InN) by using severaliteration parameters and a more complicated iterationscheme. For more general elliptic problems with variable coefficients, suitable values for these parametersmust be determined by either eigenvalue analysis ornumerical experimentation. A number of algorithms for the efficient direct solution of the discretized Poisson problem have been. introduced (e.g., Buzbee et al., 1970; Sweet, 1977;Swarztrauber, 1977). These "fast Poisson solvers" arebased on cyclic reduction, matrix decomposition, or acombination of the two; many make use of the FastFourier Transform (FFT) algorithm. While highly efficient for the Poisson problem, these techniques usually cannot be extended to more complicated problemssuch as (1.2). As we shall see, multigrid methods solvethe Poisson problem as efficiently as these fast Poissonsolvers, giving the solution in O(N2) operations, butalso generalize to solve a wide range of more complicated problems with essentially the same efficiency.3. Basic multigrid concepts In this section we motivate multigrid methods byanalyzing the convergence of Gauss-Seidel relaxation.This analysis suggests a simple two-grid method, whichnaturally generalizes to an efficient multigrid method.a. Analysis of relaxation Consider a single sweep of Gauss-Seidel relaxation,and denote the approximations to the true (discrete)solution uh of (2.4) before and after the sweep byandUacw,~h respectively. Subtracting the relaxationequation (2.7) from the discrete equation (2.3a) showsthat the corresponding errors v~ = u~ - t2~ and= uh- tJ~cw satisfyVj~, ~(Vj-l,k + Vj+~,~ + ~ew + Vj,k+l). (3.1)Thus the new error produced at a point (x~, Y0 byrelaxation is simply the average of the current errorsat the four surrounding points. This indicates that errorcomponents which oscillate on the scale of the grid willbe reduced substantially by relaxation, due to cancellation on the right-hand side of (3.1), while smootherror components will be reduced only slightly. Thusrelaxation is efficient as a smoother, but inefficient asa solution method. To quantify this result without the complexity of arigorous eigenvalue analysis one can use local modeanalysis (Brandt, 1977a). The discrete Fourier modesare Eo(x~, YO = exp[i(jO, + k02)], (3.2)where 0 = (0:, 02) is the discrete vector wavenumber;on a grid with mesh spacing h the components 0: and02 are integral multiples of 2;rh between -;r and +~r.For periodic problems, expansions in terms of themodes (3.2) are appropriate, but even in nonperiodicproblems they provide useful and quantitatively correctinformation about local processes such as relaxation.Suppose that the error vn before the relaxation sweephas a component AoEo, where Ao is the amplitude. Thenfrom (3.1) and (3.2) the error V~w~ after the sweep hasthe component AoaeWEo, with Ao and Aon"w related by I (e_iO~AoneW eiO~Ao e_iO~AonewA0new = ~ '~- + + ei-2Ao).(3.3)Thus, one relaxation sweep reduces the amplitude ofthe error component Eo by the convergence factor [Ao~'[ = [ ~(0) = I Ao [ 4 - e-~-' - e-i-~[ ' (3.4)Figure 2 shows # as a function of 01 and 02; as arguedabove, tt is small for high wavenumbers and large forlow wavenumbers. In particular, from (3.4) we have ~= % when 1011 = 102l = ~, but t~ = I - O(h2) when bothO~ and 02 are O(h). Thus the modes responsible for theslow convergence (2.10) of Gauss-Seidel relaxation arein fact the lowest wavenumbers; these modes could beadequately approximated on a coarser grid with muchless work. This observation is the key to the multigfidmethod.948 MONTHLY WEATHER REVIEW VOLUME 114FiG. 2. Convergence factor ~t(0) for the Poison problem (2.3) usingGauss-Seidel relaxation Mth lexico~phic ordering.b. A two-grid method The use of a coarser grid to approximate low-wavenumber error components can be illustrated as follows.Suppose that several relaxation sweeps have been carded out for the problem (2.4) on the grid fin, yieldingan approximate solution fla. From the discussion inthe previous section, the associated error va is smooth,i.e., the high-wavenumber modes have been reducedsubstantially, and further relaxation will reduce the remaining low-wavenumber modes very slowly. Now theerror satisfies the residual equation Lava= rh, (3.5)where rh = fh _ Lafh (3.6)is the residual; since va is smooth, (3.5) can be approximated on the coarser grid ~2H with mesh spacingH>hby Lnvn = Ianr~. (3.7)Here Ln is the same discrete operator as Lh excep~ forgrid spacing H, and vn is the grid H approximation tovn. Usually one takes H = 2h as shown in Fig. 3. Theright-hand side of (3.7) is obtained by computing theresidual r~ on the fine grid flh using (3.6) and transferring it to the coarse grid f~n, e.g., by injection (simplycopying the values of rh at points common to bothgrids); the operator I~u represents the fine-to-coarse gridtransfer. Assuming that (3.7) can be solved, the resultvn is an approximation to the error va and hence canbe transferred back to the fine grid ~2h, e.g., by bilinearinterpolation, and added to the previous approximatesolution flh to obtain a new approximation t~haew = ~a + in~vH, (3.8)where Iff represents the coarse-to-fine grid transfer. In the simple two-grid cycle described above, relaxation is used on the fine grid to efficiently reduce high.wavenumber error components, and the coarse-gridcorrection cycle (3.6)-(3.8) is used to eliminate the lowwavenumber errors. Here the "high-wavenumber"modes are the modes not representable on the coarsegrid f~n; these satisfy ~r < 101 < ~r, where ~ = h/Hand101 = max(lO~l, 1021), corresponding to the region betweenthe two boxes in Fig. 2 (assuming ~ = V2). We definethe smoothing factor ~ as the maximum convergencefactor #(0) for the high wavenumbers, i.e., ~ = max t~(0), (3.9)where 01 = ~,r has been included for convenience. ForGauss-Seidel relaxation with ~ = V2 one finds from (3.4)or Fig. 2 that ~ = V2; thus each relaxation sweep reducesthe high-wavenumber error by at least a factor of 2.Assuming that the coarse-grid equation (3.7) is solvedexactly and neglecting any errors introduced by thegrid transfers, a two-grid cycle consisting of ~ relaxationsweeps on the fine grid followed by the come-grid correction (3.6)-(3.8) reduces the error in a//wavenumbersby the factor ~. Since the grid transfers in fact usuallyintroduce a small amount of error, the accuracy cannotbe increased without limit by simply increasing ~.However, with small v the above estimate holds, andFIG. 3. The fine grid ~a and coarse grid ftt~ = 122~ for the two-grid method and their superposition.MA-1986 FULTON, CIESIELSKI AND SCHUBERT 949the two-grid cycle can be repeated as needed to attainthe desired accuracy. With v = 3 the error is reducedby almost an order of magnitude per cycle, independentofh.c. A multigrM method The method used to solve the coarse-grid equation(3.7) was left unspecified in the two-grid cycle describedabove. However, since this equation has the same formas the original equation (3.1), it can be solved by thesame method; i.e., make several relaxation sweeps onthe coarse grid ~n and then apply a correction computed using a still coarser grid fl2H- Continuing thisprocess recursively leads directly to a simple multigfidcycle. More precisely, we define a set of grids G/ = fl*t(l = 1 ..... M) with mesh sizes hl satisfying hl-~ = 2hi.The finest grid G3~ has the desired resolution hM = h,while the coarsest grid G, has only a few points in eachdirection; the number of grids M is typically about fiveor six. Changing notation slightly, for each grid Gl (oftenreferred to as "level" l) we form a discrete problem Llul = fl, (3.10)where L/ is the finite difference approximation to theoperator L = -V2 for mesh size hi. For the finest gridI = M, (3.10) is simply the original discrete problem(2.4) with uM = u~ and fM = f~. However, for eachcoarser grid (3. I0) is an approximation to the residualproblem on the next finer grid, so that for I < 1 ~< M,ul-~ andfI-t are approximations to the error vI = ul- ~/and residual rI = f/- Lt~/, respectively (where ~/denotes the current approximation to the exact discretesolution ul). Thus vg and IhUr* in the two-grid methodhave been replaced by ut-~ andf/-~ for uniformity ofnotation. The multigrid cycle, shown schematically in Fig. 4,starts on the finest grid G~ with an approximation ~Mto /he desired solution u~. For each level l = M,M- 1 ..... 2 in turn, the corresponding problem (3.10)is relaxed v~ times to smooth the error on the scale ofgrid Gt, and the resulting residual / = fl _ Ll~/istransferred to the next coarser grid Gl-~ via f/-' = I/-'r/= I/-'(ft- L/a/) (3.11)to form the fight-hand side of the level l - I problem(3.10). In this procedure the initial approximations ~lon the coarser grids Gi (1 ~< I < M) are simply takento be zero. Upon reaching the coarsest grid Gt the resulting problem L~u~ = f~ is simply solved, either directly (by Gaussian elimination) or by many relaxationsweeps; either approach is efficient since the coarsestgrid contains very few points. Then for each level l = 1,.... M - 1 in turn, the current approximate solutiont~/, being an approximation to the error u/+~ - ~l+~ onthe next finer grid, is interpolated to that grid and addedto the previous approximation via Grid%:Ge G$ G,Mesh Size h~=h h5 = 2h h4 = 4h h~ = 8h h~: 16h h~ = 32hn Relaxation Sweeps ~: Transfer ResidualSolve / : Interpolate Correction FIG. 4. A multigrid V-cycle for M = 6 grids. ~/+1 = /~1+1 + I/+l~l, (3.12)and the level l + 1 problem is relaxed v2 more timesto smooth any remaining errors on the scale of thatgrid. The overall cycle from the finest grid to the coarsest and back is called a multigrid V-cycle. To estimate the effect of a multigrid V-cycle, notethat each Fourier mode 0 on the original grid f~a is infact a high-wavenumber mode on one of the grids used.Thus the error in each mode is reduced by at least thefactor ~ by v = Vl + v2 relaxation sweeps on one ofthe grids during one V-cycle. If v is not large (typicallyone might choose h = 2 and v2 = 1), the errors introduced by the grid transfers are negligible, so the overallerror (i.e., the error in all modes) is reduced by at leastthe factor ~ per V-cycle. This convergence factor isindependent of h, unlike the factors (2.10) for classicaliterative methods which deteriorate as the grid resolution increases. Therefore, reducing the overall errorby a specified factor ~ requires s = lm/(v ln~) V-cycles,independent of h. Furthermore, since each grid contains only one-fourth as many points as the next finergrid, the total number of grid points (and hence storage)involved is only 1 1 1 4 ... 5times that required for the original problem (3.1) onthe finest grid. Therefore, the work required per Vcycle (ignoring the overhead of residual transfers andinterpolation, which is usually small) is less thantimes the work required for v relaxation sweeps on thefinest grid alone. Thus with O(N2) points on the finestgrid the work per V-cycle is O(N2) operations; sincethe number s of V-cycles required is independent of h950 MONTHLY WEATHER REVIEW VOLUME 114(and hence N), the multigrid method reduces the errorby a specified factor in O(N2) operations.4. Survey of multigrid techniques In this section we discuss in more detail the variouselements of a multigrid method for a problem of thegeneral form Lu =f, (4.1)where L is an elliptic partial differential operator (typically in two or three space dimensions), u is the unknown, andfis specified. These elements include therelaxation scheme, grids, grid transfers, and control algorithm. In most of the discussion we assume for simplicity that L is linear; the Full Approximation Scheme,which can be used to treat nonlinear problems, is alsodescribed. Since the boundary conditions do notstrongly influence the essential features or efficiency ofa multigrid method, their treatment is discussed separately. More detailed information on all of the topicsreviewed here can be found in Brandt (1984).a. Relaxation schemes The heart of a multigrid method is the relaxationscheme used on each grid to smooth the error, i.e., toreduce the high-wavenumber error components. Therelaxation scheme is the most problem-dependent partof a multigrid method, and has the most impact onthe overall efficiency. Since smoothing high-wavenumber errors is essentially a local process, it may beanalyzed using local mode analysis, and constant (frozen) coefficients may be assumed in the analysis forproblems with variable coefficients (unless they varydrastically on the scale of the grid). The primary measure of the effectiveness of relaxation is the smoothingfactor ~ defined by (3.9); small ~ is desired for fastsmoothing. In addition, a relaxation scheme should besimple (relatively few operations per grid point for efficiency) and robust (relatively insensitive to changesin parameters or coefficients). The simplest relaxation schemes are the successive(Gauss-Seidel) schemes, in which the new values ofunknowns replace the old values immediately afterbeing calculated. These schemes are appropriate formost elliptic problems. For example, ordinary (point)Gauss-Seidel relaxation with lexicographic orderinggives ~ = 0.5 for the Poisson problem, as shown above.This scheme is simple, involving only four additionsand one multiplication per grid point. The smoothingfactor ~ = 0.5 is not improved by introducing a relaxation parameter; in contrast, when relaxation is usedas a solver, faster convergence is obtained by overrelaxation (i.e. the SOR method). For anisotropic problems, point GaussoSeidel relaxatioh may not be as effective. For example, discretizingthe equation (1.2) using centered differences on a gridwith mesh spacing h yieldsh-2[aj+W2,~(~td+l,k -- Uji-) -- ad-ll2,~(Ujk -- Ud-l,k) (4.2)where aj+w2,k = a(U + 1/2)h, kh) and so forth. PointGauss-Seidel relaxation for (4.2) Mth lexico~aphic ordering, defined byh [aj-l/2,kUj-l,k + aj+ll2,k~j+l,k + bj, k-w2 + bj,~+,a)fif/TM] =~kl (4.3)yields the convergence factor ] ae'-'+be'" ~(0) = 2(a + ~ ~ ae-i~' - bCi~ (4.4)using local mode analysis with a and b frozen. Whena = b (i.e., the problem is isotropic), (4.4) reduces to(3.4) so that ~ = 0.5 as for the Poisson problem, butwhen a and b ~ffer, ~ can ~ considerably l~ger. Fibre5 shows ~(0) for the case b/a = 10, for which ~ ~ 0.83;as b/a ~ ~ (or a/b ~ ~), ~ ~ 1. The difficulty here is that point rel~ation smoothsthe e~or only in ,~rections of strong local coupling.This can be seen by noting from (4.2) and (4.3) thatthe e~ors v~n and v~TM satisfyv~. = a(v~,7~ + v~+~,~) + b(v~:~ + v~.~+t) 2(a + b) (4.5)Oao e~~l 0'$-~..~ i i ~r o ~r -~r 'T 2 0~FiG. 5. Convergence factor ~(/?) for problem (4.3) usingGauss-Seidel relaxation with lexicographic ordering and b/a = 10.MA-1986 FULTON, CIESIELSKI AND SCHUBERT 951When b/a >> I the solution is coupled more stronglyin y than in )c, cancellation between errors at O' - 1,k) and (j + 1, k) will have little effect in (4.5) so thathigh wavenumbers in x will be changed only slightlyby relaxation. Since point relaxation is slow to convergein this case, locally strongly coupled blocks of pointsshouM be relaxed simultaneously. For the case b/a >> 1,y-line Gauss-Seidel relaxation is appropriate; thisscheme is defined the same as (4.3) except with fi2,~+~replaced by fi.~,[g~, so that all points along a y-line (constant x) are relaxed simultaneously. Relaxing each yline in turn is easy and efficient, since the linear systemsinvolved are tridiagonal and diagonally dominant. Theresulting convergence factor t40) = 2(a + b - b cos~2) - ae-~ (4.6)is shown in Fig. 6a for the case b/a = 10. It can beshown from (4.6) that ~ = max 5-m, (4.7)so that ~ m 0.45 for any b/a ~> 1. However, (4.7) showsthat ~ ---* 1 as b/a -~ O, as can be seen in Fig. 6b whichshows/~(0) for the case b/a = 1/10. This case requiresx-line relaxation by the same reasoning. When b/a islarge in some regions and small in others, one can usealternating direction line relaxation, consisting of onesweep of y-line relaxation followed by one sweep ofx-line relaxation. Such a double sweep gives ~ ~< 5-1/2~ 0.45 for any ratio b/a. This method is similar to theADI solution method, but much simpler since thereare no iteration parameters to be chosen. The Gauss-Seidel schemes discussed above are notdirectly vectorizable and hence do not exploit the fullpotential of vector processors such as the CRAY-1 orCYBER 205. Simultaneous (Jacobi) relaxationschemes, in which all of the new values in a singlesweep are computed before overwriting any of the oldvalues, would seem preferrable since they do vectorize.However, in addition to using more storage, Jacobischemes usually require underrelaxation (i.e., using arelaxation parameter w < 1) and thus more work pergrid point, and have poorer smoothing factors (e.g., ~= 0.6 with w = 4/5 for the Poisson problem). A betterapproach is to use Gauss-Seidel schemes with the gridpoints or lines taken in a different order. For isotropicproblems one can use point Gauss-Seidel relaxationwith red-black ordering: first relax at all "red" points(j + k even) and then at all "black" points (j + k odd),where the colors refer to the pattern of a checkerboard.Similarly, when line relaxation is required one can usezebra relaxation: first relax along all "white" (even)lines and then along all "black" (odd) lines. Each ofthese schemes is fully vectorizable. Moreover, they havebetter smoothing factors than the correspondingschemes with standard ordering. (Their mode analysis (a)lr01 00,2 ~0.1~I I I.*r 0 ~ ~r2 2 8~ (b) O__0 6 5 .4-' 1//~.5 , ; -~r -~- 0 ~2 e~F~G. 6. Convergence famor ~(8) for problem (4.3) using ~lineGau~Seidel rel~afion for (a) b/a = 10 and (b) b/a = 1/10.O, 0is more complicated, however, since they couple severalmodes.) For example, point Gauss-Seidel relaxationgives ~ = 0.25 with red-black ordering, as opposed to~ = 0.5 with lexicographic ordering; this excellentsmoothing factor makes the red-black scheme the relaxation method of choice for the Poisson problem.For anisotropic problems with variable coefficients, alternating direction zebra relaxation is robust and efficient.952 MONTHLY WEATHER REVIEW VOLUME 114b. Grids A multigrid method for (4.1) typically uses a gridstructure much like that described in section 3c: a setof grids G~, G2 ..... G~u with mesh sizes hi > h2 > - - -> hM, each grid covering the whole computational domain. For convenience of notation it is assumed thaton each grid the same mesh size is used in each coordinate direction, although this need not be the case.The mesh size ratio ht+ ~/hi is usually the same constantb between all grids; this ratio affects the efficiency intwo ways. First, b must be relatively small so that thecoarse grids Gi ..... GM-i involve relatively few pointscompared to the finest grid GM, and thus do not contribute substantially to the total work or storage. Second, ifb is too small then relaxation becomes inefficientas a smoother, since more modes must be consideredas high wavenumbers in (3.9). For these reasons, thecomputationally convenient ratio ~ = 1/2 is usuallyoptimal, although ~ = 1/3 or ~ = l/d~ (with rotatableoperators) may be useful in some cases. The basic grid structure described above may bemodified in several ways. First, a gix;en grid G~ may becoarser than the next finer grid Gt+~ in only some ofthe coordinate directions; this semi-coarsening canprovide an alternative to block relaxation. For example,for the three-dimensional problem Lu = au~ + bu~+ CUzz = fwith 0 < a ~ b ~ c, the strong coupling iny and z requires y-z plane relaxation, which is ratherawkward. An alternative approach is to coarsen thegrids in y and z only, and use z-line relaxation pointby point in x and y. This approach is economical inthree dimensions since the coarse grids still containrelatively few points. Second, the grids do not all need to cover the wholeproblem domain. (This requires the Full Approximation Scheme described below.) In this way nonuniformresolution can be obtained, e.g., more resolution nearsingularities and sharp gradients and less resolution inouter regions to simulate unbounded domains. Programming is relatively easy since all operations are stillperformed on uniform grids. Local refinements can alsobe made adaptively, i.e, as required by the numericalsolution as it evolves, leading to what Brandt refers toas Multi-Level Adaptive Techniques (MLAT).c. Grid transfersAssociated with each grid Gt is a discrete problemLtu~ = f~, (4.8)where Lt is a discrete (finite difference) approximationto L for mesh size h~. For linear problems with coextensive grids, these discrete problems are interpretedas described in section 3c. That is, on the finest grid(l = M), (4.8) approximates the original problem (4.1),while on each coarser grid (l < M), (4.8) approximatesthe residual problem for the next finer grid. This interpretation is called the Correction Scheme (CS), sinceut-~ is a correction to u( Two types of grid transfersare required: the fine-to-coarse transfer of the residualr~ = ft - Li~t via (3.11), represented by the operatorI/-~, and the coarse-to-fine interpolation of the correction via (3.12), represented by I/+t. The proper choiceof grid transfers depends in part on the order of thederivatives in L; precise rules are given in Brandt (1984,section 4.3). For most second-order problems one canuse linear interpolation in each coordinate directionfor the coarse-to-fine transfers. The choice of appropriate residual transfers, however, is somewhat moreinvolved. When the residual is smooth after relaxation, it mayusually be transferred to the coarse grid by injection,i.e., simply copying the values to the coarse grid at thepoints common to both grids. In some cases, however,the residual is not smooth after relaxation, even thoughthe error, is. For example, with red-black or zebra ordering of relaxation, the residual is zero at the set ofpoints most recently relaxed and nonzero elsewhere;nonsmooth residuals also can occur when Lt has nonsmooth coefficients. In such cases injection does notgive an adequate approximation to the residual, andfull weighting should be used instead. In this procedurethe residual value transferred to a coarse-grid point isa weighted average of residual values at surroundingfine-grid points, with the weights chosen to preservethe integral of the residual over the problem domain.For example, 9-point full weighting in two dimensionsis given by i[_~r~ = ~l,m 4 r~, (4.9) 2where the weights are applied in the pattern indicated.As in all residual transfers, if the discrete problem hasbeen multiplied by a power of ht this m.ust be takeninto account: the residual to be transferred must bebased on the divided form of Lt, i.e., the form whichdirectly approximates L.d. Control algorithms A multigrid method requires an algohthm whichdetermines when to switch from one grid to another.Such algorithms fall into the two basic classes discussedin this subsection. 1) CYCLING ALGORITHMS A multigr~d cycle for a discrete problem on a givengrid G~ starts with an initial approximation to the solution on that grid and ends with an improved approximation on the same grid. A cycle uses relaxationfor the Gi problem and corrections computed by transferring the G~ residual problem to the next coarser gridMAY 1986 FULTON, CIESIELSKI AND SCHUBERT 953G/_~; the G/_~ problem in turn is also solved approximately by one or more multigrid cycles (unless Gt-~ isthe coarsest grid, in which case it is solved either directlyor by many relaxation sweeps). Multigfid cycles canbe used as a solution method by starting on the finestgrid G~r with an initial approximation to the solutionand making cycles repeatedly until the desired accuracyis obtained. Multigrid cycles can be either fixed or accommodative. That is, the decisions of when to switch gridsand which direction to go (i.e., to a coarser or finergrid) can be either prescribed in advance or made internally as the numerical solution develops. The Vcycle described in section 3c is a simple example of afixed multigrid cycle. Other examples include the Wand F-cycles described in Brandt (1984); these solvethe coarse-grid problems more accurately than Vcycles, and hence are more robust but often less efficient. Such fixed multigrid cycles are especially desirable during the development ofa multigrid code, sinceaccommodative algorithms can obscure conceptual orprogramming errors by their more complex internalinteractions. On the other hand, accommodative algorithms are more robust. The cycle-C algorithm of Fig. 7 is a generally applicable, fully accommodative cycling algorithm. The tlowcan be traced starting with the box labeled fi/~- Relax[L~ . = ft]at, a shorthand notation for "make one relaxation sweep on the level l problem Llul = ft to improve the current approximation t~t''. After a relaxationsweep, ~t is tested to see whether it has converged. Ifit has, the level l problem has been solved, so if I < M J rI&~'Conve,0,d~ No Yes I ~=MFIG. 7. The multigfid cycle-C algorithm (after Brandt, 1977a).its solution is interpolated to the next finer level l + 1and used as a correction to the current approximationt~t+~ there. On the other hand, if/~l has not converged,the speed of convergence is checked. Fast convergenceindicates that the error is not smooth yet (i.e., substantial high-wavenumber errors still exist); relaxationis still effective, so another sweep is made. Slow convergence indicates that the error is smooth (i.e., highwavenumber errors have been reduced substantiallyby relaxation); further relaxation would be ineffective,so the residual is transferred to the next coarser levell - 1 to serve as the right-hand side of the discreteproblem there, and the initial approximate solutionfi~-~ is set to zero. On the coarsest grid l = 1, relaxationcontinues until the solution converges. The cycle startson the finest grid l = M with an initial approximationfi~ to u~, and ends on that grid when ~t has converged. In an accommodative algorithm the tests of convergence are generally made in practice using dynamicresiduals, i.e., residuals computed during the relaxationprocess. For example, point Gauss-Seidel relaxationwith lexicographic ordering (2.7) for the Poisson problem can be written as h2 a~w = aj~ + ~- r~, (4.10)where~ = f~ - h-214~j~ -- a~-e~V,k- aj+l,k- a~,~w~, -- aj, k+~l (4.11)is the dynamic residual at the point (j, k). The speedof convergence can be tested by computing the ratioof some norm (e.g., rms or max norm) of the dynamicresidual associated with the current relaxation sweepto the corresponding residual norm for the previoussweep. If this ratio is dose to the smoothing factor ~(or smaller), the convergence is fast and relaxation isstill effective; if it is significantly larger than ~, the convergence is slow and relaxation is no longer effective.Similarly, convergence is obtained on a given grid G~when the residual norm is less than a specified factor~ times the latest residual norm on the next finer grid,i.e., the accuracy of the Gt+~ solution (for which it is acorrection). The performance of the algorithm is notvery sensitive to the choice of the parameter ~; Brandt(1984) suggests the choice ~ = 2-~, where d is the number of dimensions of the problem. On the finest grid l = M one usually solves "to thelevel of truncation error", i.e., so the norm of the residual r~ = f~t _ L~t2~ (or the dynamic residual) iscomparable to that of the truncation error r~u = L~([~Uu) - f~, (4.12) where u is the solution of the continuous problem (4.1) and [~u represents a continuum-to-grid M transfer of the solution. Further algebraic convergence of flu to u~ is usually not needed, since it would not necessarily954 MONTHLY WEATHER 'REVIEW VOLUME 114make ~M any closer to u. An estimate of the norm ofthe truncation error can be specified in advance (usingTaylor expansions and typical values of derivatives ofu), or can be generated internally during the multigridprocess if the Full Approximation Scheme (FAS) isused. 2) FULL MULTIGRID (FMG) ALGORITHMS When using multigrid cycles as a solution methodone starts with an initial approximation tim on the finestgrid GM. One possible choice is simply tim = 0; however, if abetter initial approximation can be made thenfewer cycles will be required to solve the problem. Toget a better first approximation one can first solve thediscrete approximation to (4.1)--not a residual problem-on the next coarser grid GM_~, and interpolatethat solution to grid GM. Extending this idea recursivelyback through coarser and coarser grids leads to a FullMultigrid (FMG) algorithm. Figure 8 illustrates a simple FMG algorithm for aproblem with M = 4 grid levels. The process starts onthe coarsest grid l = I where the discrete approximationto (4.1) is simply solved. The resulting solution is interpolated to the next finer grid l = 2 and used as thefirst approximation to the solution of the correspondingG2 problem, which is then solved by a multigrid Vcycle. This process is repeated, interpolating, the solution from one level to the next as a first approximationand solving by a V-cycle, until the final solution is obtained on the finest level l = M. Thus the FMG algorithm is, in fact, a nested iteration method, i.e., a"bootstrap" method which uses an iterative solvereach grid G~ through G~ in turn; the iterative solverused for'each gridis a multigrid cycle. Although Fig.8 shows simple V-cycles, any.multigrid cycle, fixed oraccommodative, can be used. The initial interpolationto a finer grid (denoted by a double arrow in Fig. 8)may be different than that used in the multigrid cycles;in particular, it generally should have a higher orderof accuracy, since the function being interpolated isrelatively smooth (Brandt, 1984, sec. 7.1). FMG algorithms are very efficient. In going fromone level I to the next finer level l + 1, the truncationerror decreases by roughly the factor ~', where b = ht+~ /h/is the mesh ratio and p is the order of the differenceapproximations. Since a single multigrid cycle typicallyreduces the error by at least this factor, an FMG algorithm which uses just a single cycle per grid levelwill give the final solution on each level. (denoted bythe double circles in Fig. 8) to at least the level of therespective truncation error. The computational workexpended is usually expressed in terms of the multigridwork unit, defined as the amount of computationalwork required per relaxation sweep on the finest grid.For the algorithm of Fig. 8 the total work required(assuming a two-dimensional problem with ~ = 1/2 andignoring the work of grid transfers) is 2 3 M ) 16v I +~+~+ -.. +4-D:7 <~-v (4.13)work units, where v = v~ + v2. Typically v = 2 or v = 3is suff~cient, so only 3.6-5.3 work units are required.This efficiency is typical: optimal multigrid methodsfor most problems yield the solution to the level oftruncation error in only a few (4-10) work units. GridGM = G4 Mesh Size h4= hGa h~ = 2 hGz h2 = 4hGI h~ = 8hG:lqG. 8. n Reloxotion Sweeps ~ Tronsfer ResiduolSolve~ Inferpolote CorrectionSolution Converged I Initiol InterpolotionA Full Mulfigrid (FMG) algorithm for a problem with M = 4 grids.M^YI986 FULTON, CIESIELSKI AND SCHUBERT 955e. The Full Approximation Scheme (FAS) The multigrid methods described so far use the correction scheme: each coarse-grid equation is an approximation to the residual problem on the next finergrid, so the coarse-grid solution is an approximate correction to the fine-grid solution. This scheme ~s validfor linear problems only, since it depends on the linearity of L to write the residual problem for the correction vn = uh - fin as Lh~9h = rh = fh _ Lhfh. Similarly, the correction scheme requires coextensive grids,since the coarse grid cannot provide a correction tothe fine-grid solution where the fine grid does not exist.An alternative scheme which circumvents these restrictions is described below.- Without assuming linearity, the fine-grid residualequation Ln(fh + vn) _ Lhfn = rh = fn _ Lnfh (4.14)can be approximated by the coarse-grid equationLH(.[hHfh -t- 'OH) -- LH(I~hHfh) = IhHrh. (4.15)Here, vH approximates the true correction Vh to fin,and the solution transfer operator inu may differ fromthe residual transfer operator Ian. Putting the knownquantities on the right reduces (4.15) to Lntiu = f~, (4.16)where tin = [hnfh _[_ VH (4.17)and fn = Ln(innan) + ihnrh. (4.18)This is known as the Full Approximation Scheme(FAS), since from (4.17) the coarse-grid variable tinapproximates the full solution ua, not just the correction va. After solving (4.16), the fine-grid solution isupdated via f~e,~ = t2a + Inh(t~~ - [nnt~h), (4.19)which represents interpolating and adding the approximate correction vn. Note that (4.19) cannot be simplified to fhnew = [,htiH without losing all high-wavenumber information in Using the FAS, multigrid methods can solve nonlinear problems with the same efficiency as the correSponding linearized problems. No global linearizationis needed: a single Newton iteration often suffices tosolve the nonlinear relaxation equation at a singlepoint, but even this simple local linearization is usuallyunnecessary. In addition, FAS allows local mesh refinement for obtaining nonuniform resolution organized by simple uniform grids, as mentioned in section4b. Where the fine grid exists, the coarse-grid variablerepresents t~n as defined by (4.17); where the fine grid.does not exist, the coarse-grid variable simply representsthe coarse-grid solution. The FAS also provides a useful analogue of the localtruncation error as follows. From (4.12), the truncationerror rn on the coarse grid satisfies Ln([nu) = Inf + rn (4.20)and thus rn = Ln(lfnu) - In(Lu), (4.21)where in and In represent continuum-to-grid H transfers of the solution and fight-hand side, respectively.Thus rH is the quantity which must be added to thecoarse-grid equation so that its solution coincides (onthe coarse grid) with the continuous solution. Similarly,substituting from (4.18) the FAS coarse-grid equation(4.16) becomes Lntin = Ihnfh + rhtt (4.22)where yaH = LH([hnfh) -- Ihn(Lhfh). (4.23)At convergence, tin = [hnuh, SO rh~ is the relative localtruncation error, i.e., the quantity which must be addedto the coarse-grid equation so that its solution coincides(on the coarse grid) with the fine-grid solution. The computable analogue rhn to the truncation errormakes possible several useful techniques. First, rhH canbe used to raise the order of approximation on thecoarse grid; this r-extrapolation is similar to Richardsonextrapolation but more general, since it extrapolatesthe equation, not the solution. Second, rnncan be usedto estimate the truncation error rH (or ra), thus providing criteria for stopping multigrid cycling or foradaptive local mesh refinement. These and other applications of rnn are discussed in Brandt (1984, sections8 and 9).f Bounclary conditions Ideally, boundary conditions should not affect theoverall efficiency off multigrid method (assuming, ofcourse, that the problem is well-posed). The boundarytypically contains far fewer grid points than the interiorof the problem domain; if extra work per grid pointmust be invested near the boundary it will not increasethe total computational work by much. However, withcomplicated boundary conditions one must design eachelement of the multigfid method carefully in order toattain this ideal. Dirichlet conditions (i.e., u specified along theboundary) are easily incorporated into multigridmethods. Each time an initial approximation is formedon a grid, the known boundary values are inserted directly. In correction schemes, this means using thespecified values on the boundaries of the finest gridand zeros on the boundaries of the coarser grids. Onlythe interior equations are relaxed, so the boundaryconditions are satisfied on all grids at all times. More general conditions, such as Neumann conditions (i.e., the normal derivative of u specified alongthe boundary) can be implemented, keeping in mindthe following two basic principles. First, boundary956 MONTHLY WEATHER REVIEW' VOLUME I14conditions are equations and hence must be relaxed,just as the interior equations are relaxed; their residualsafter relaxation must be transferred to the coarse gridalong with the interior residuals. These residuals mustbe based on the divided form of the equations, i.e.,with the finite difference approximations to derivativesnot multiplied through by a power of h. Second, therole of boundary relaxation is not to impose theboundary conditions but to smooth the error along theboundary (similar to the role of interior relaxation).Thus, relaxation should change a boundary value notso that the corresponding boundary condition is satisfied exactly but so that the resulting error is the average of the errors at neighboring boundary points.g. Generalizations The basic multigrid idea of cycling between different levels of discretization to reduce the error efficiently on all scales can be applied to discretizations other than finite differences. For example, series approximation methods (finite element and spectral) represent the un known as a truncated series of known basis functions. Multigrid (or, more properly, multilevel) techniques for such discretizations use a set of approximation sub spaces S~, S2 ..... SM in place of the grids Gi, G2, .... G~u, The transfer operations between levels are usually very natural; for example, since St-~ is usually a subset of& the coarse-to-fine transfer It_~~ is simply the identity operator. With finite-element discretiza tions (i.e., local basis functions) most aspects of the multigrid process are similar to the finite difference case (e.g., Brandt, 1977a, sections 7.3 and A.5; Nico laides, 1977; Brandt, 1980; Bank, 1981). With spectral or pseudospectral discretizations (i.e., global basis functions) the relaxation scheme must be designed carefully to take advantage of fast transform techniques (Zang et al., 1982, 1984). The multigrid idea can. also be used 'in solving discrete problems which have no underlying geometric. interpretation. Algebraic multigrid (AMG) methodssolve matrix problems of the form Ax -- b, where A ispositive definite, using only the information in the matrix A to construct an appropriate set of coarser "levels'', each consisting of a subset of the original variablesin x (Stfiben, i983). Such methods exhibit the usualmultigrid efficiency, i.e., solution in just a few workunits; however, they require considerable overhead interms of storage and set-up time. They are useful whenthe construction of conventional (geometric) multigridmethods would be impossible (e.g., no underlying geometry) or very difficult (e.g., nonuniform finiteelement discretizations). Multigrid methods can be generalized in many other ways, e.g., to nonelliptic problems, eigenvalue prob lems, and systems of equations; some of these gener alizations are mentioned briefly in section 5. However,a discussion of the techniques involved is outside thescope of this review, and the interested reader is referredto the multigrid literature cited, especially Brandt(1984), for details.5. Applications Multigrid methods have been applied successfullyto a wide range of problems. Much of this work is quiterecent; some methods are still being refined, and newapplications are currently being demonstrated. Thissection briefly surveys some of the more prominentapplications to date, and points out areas of currentresearch. Multigrid methods were originally developed as fastsolvers for discretized elliptic boundary-value problems. The multigrid treatment of the Poisson problemwas discussed in detail by Stfiben and Trottenberg(1982). Gary et al. (1983) compared multigrid, SOR,and conjugate gradient methods for the diffusionequation -V. (DVu) = f, (5.1)where D is a diffusion coefficient (scalar or tensor),treating the three-dimensional case with D assumed tovary slowly. In some applications, such as neutron diffusion, D may have strong discontinuities (i.e., jumpsof several orders of magnitude); this more difficult casehas been treated in two dimensions by Alcouffe et al.(1981) and in three dimensions by Behie and Forsyth(1983). Kettler (1982) studied multigrid methods forthe convection-diffusion equation--V- (DVu) + v. Vu = f(5.2)in two dimensions, giving detailed analyses of smoothing properties for many different relaxation schemes.Singular-perturbation problems have also been treated(Brandt, 1979; Hemker, 1982); efficient multigridmethods for such problems rely heavily on appropriatelocal mesh refinements. Various types of nonlinear problems have also beensolved by multigrid methods. Among mathematicians,a favorite nonlinear test problem isheu - V2u = f, (5.3)which arises in chemical reaction theory. Multigridmethods for (5.3) have been developed by Meis et al.(1982); current research includes continuation techniques for this and other bifurcation problems (Mittlemann and Weber, 1985). Ordinary (linear) eigenproblems can also be regarded as nonlinear, since theunknown eigenvalue multiplies the unknown eigenfunction. An FAS-FMG method obtains each eigenfunction with the usual multigrid efficiency (Hackbusch, 1979; Brandt et al., 1983).M^Y1986 FULTON, CIESIELSKI AND SCHUBERT 957 Much research has been done on applying multigridmethods to more complicated nonlinear problems influid dynamics, including systems of equations. Theequation governing transonic potential flow (e.g., airover an airfoil) is not only nonlinear, but also ellipticover parts of the domain and hyperbolic elsewhere. Ithas been solved efficiently by multigrid methods in twodimensions (Boerstoel, 1982), three dimensions(Caughey, 1983), with mesh refinement (McCarthy andSwanson, 1983), and with finite-element (Deconinckand Hirsch, 1982) and spectral (Streett et al., 1985)discretizations. Multigrid methods have also been developed for the Euler equations (e.g., Jespersen, 1983)and the Navier-Stokes equations (e.g., Ghia et al.,1982); a detailed summary of techniques for these systems appears 'in Brandt (1984). Most applications influid dynamics so far have been for steady state solutions; multigrid methods for time-dependent flows havebeen discussed by Brandt et al. (1980), Lee and Myers(1980), Jameson (1983), and Johnson (1983). Research in basic multigrid techniques also continues. One area of interest is higher-order techniques.Schaffer (1984) investigated higher-order differenceschemes, defect correction methods, and r-extrapolation as means of obtaining high accuracy with multigridmethods. Brandt et al. (1985) developed improved relaxation schemes for Fourier-spectral multigrid methods for isotropic and anisotropic problems. Mesh generation and refinement techniques have also seen muchattention (e.g., Van Rosendale, 1983; Bai and Brandt,1984). With the increasing importance of vector andparallel processors such as the CRAY- 1, CRAY X-MPand CYBER 205, research on vectorizable and parallelizable multigrid algorithms has expanded. Brandt(198 I) gives a comprehensive discussion of such algorithms, and Barkai and Brandt (1983) discuss thedetails of implementing a multigrid Poisson solver onthe CYBER 205. The relative complexity of multigrid methods mayhave been a factor in their relatively slow acceptanceup until now. Recognizing this, many researchers haveattempted to develop "black box" solvers, i.e., softwarewhich can be used as a tool by other researchers withouthaving to deal with the multigrid details. Dendy (1982,1983) has developed such routines for various types ofproblems; similar work has been done by Wesseling(1982) and Hemker et al. (1983). Foerster and Witsch(1982) describe a package of routines (MG00) for solving second-order elliptic boundary value problems onrectangular domains; listings of some routines fromthis package appear in Stiiben and Trottenberg (1982),and further details appear in Stfiben et al. (1984). Thelatter paper also describes a code (AMG01) based onperhaps the ultimate "black box" approach, namely,the algebraic multigrid concept mentioned in subsection 4g. Many of the solvers described above are available on a tape ofmultigrid software (MUGTAPE) fromthe Department of Applied Mathematics at the Weizmann Institute of Science, Rehovot 76100, Israel. Thistape also contains code for simple model problemsfrom Brandt (1977a) and Brandt (1984), plus a packageof general grid manipulation routines (GRIDPACK)described by Brandt (1977b) and Brandt and Ophir(1984). In addition, the code MGO0 has been incorporated into the ELLPACK system of software forsolving elliptic problems (Rice, 1984).6. Concluding remarks The multigrid approach is a powerful, flexible wayto develop fast solvers for discretized elliptic boundaryvalue problems. This power comes at the price of somecomplexity, but some general software is available, inaddition to detailed information on developing specificmultigrid codes. As discussed in section 1, many ellipticproblems arise in geophysical fluid dynamics, often involving variable coefficients or nonlinearities. Eventhough high efficiency is often crucial, these problemsare currently being solved by methods which are lessthan optimal. We hope this review will encourage others to try multigrid methods in such cases. In addition,multigrid methods can be combined with adaptive discretization in so-called Multi-Level Adaptive Techniques (MLAT); while still in their infancy (especiallyfor hyperbolic problems), these techniques hold considerable promise for the efficient and accurate solutionof geophysical problems involving localized phenomena and widely differing scales of motion. Acknowledgments'. The a, uthors gratefully acknowledge the help and guidance provided by Achi Brandt,especially during his sabbatical year at the Institute forComputational Studies at Colorado State University.We have also benefitted from discussions with KlausSt~iben, Gerald Taylor, and Thomas Zang, and thankOdilia Panella for her help in preparing the manuscript. This work was funded by the Naval EnvironmentalPrediction Research Facility, Monterey, California,under program element 61153N "Multigrid EllipticSolver Development."REFERENCESAlcouffe, R. E., A. Brandt, J. E. Dendy, Jr. and J. W. Painter, 1981: The multi-grid method for the difusion equation with strongly discontinuous coefficients. SIAM J. Sci. Stat. Comput., 2, 430 454.Bai, D., and A. Brandt, 1984: Local mesh refinement multilevel tech niques. 45 pp. [Available from Department of Applied Mathe matics, Weizmann Institute of Science, Rehovot 76100, Israel.]Bank, R. E., 1981: A multi-level iterative method for nonlinear ellipticequations. Elliptic Problem Solvers. M. Schultz, Ed., AcademicPress, 1-16.Barkai, D., and A. Brandt, 1983: Vectorized multigrid Poisson solver for the CDC CYBER 205. Appl. Math. Comp., 13, 215-227.958 MONTHLY WEATHER REVIEW VOLUME 114Behie, A., and P. A. Forsyth, Jr., 1983: Multi-grid solution of three dimensional problems with discontinuous coefficients. Appl. Math. Comp., 13, 229-240.Boerstoel, J. W., 1982: A mulfigrid algorithm for steady transonic potential flows around aerofoils using Newton iteration. J. Comput, Phys., 48, 314-343, Brand, K., 1982: Multigrid bibliography. Multigrid Methods. W. Hackbnsch, U. Trottenberg'Eds., Lecture Notes in Mathematics, Vol. 960, Springer-Verlag, 631-650. [Continued in Multigrid Newsletter, available from S. McCormick, Department of Mathematics, Box 170, University of Colorado of Denver, 1100 14th Street, Denver, CO 80202.]Brandt, A., 1973: Multi-level adaptive technique (MLAT) for fast numerical solution to bounda~ value problems. Proc. Third Int. Conf. Numerical Methods in Fluid Mechanics, Paris, 1972, Lecture Notes in Physics, H. Cabannes and R. Temam, Eds., Springer-Verlag, 18, 82-89.--, 1977a: Multi-level adaptive solutions to boundary-value prob lems. Math. Comp., 31, 333-390. , 1977b: Multi-level adaptive techniques (MLAT) for partial differential equations: Ideas and software. Mathematical Software III. J. R. Rice, Ed., Academic Press, 277-318. , 1979: Multi-level adaptive techniques (MLAT)' for singular perturbation problems. Numerical Analysis of Singular Pertur bation Problems. P. W. Hemker and J. J. H. Miller, Eds., Ac ademic Press, 53-142. , 1980: Multi-level adaptive finite-element methods: 1. Varia tional problems. Special Topics of Applied Mathematics. J. Frehse, D. Pallaschke and U. Trottenberg, Eds., North-Holland, 91-128. , 1981: Mulfigrid solvers on parallel computers. Elliptic Problem Solvers. M. Schultz, Ed., Academic Press, 39-83. , 1982: Guide to Multigdd Development. Multigrid Methods. W. Hackbnsch and U. Trottenberg, Eds., Lecture Notes in Mathematics, Vol. 960, Springer-Verlag, 220-312. , 1984: Multigrid Techniques: 1984 Guide with applications to fluid dynamics, 176 pp. [Available from GMD, Postfach 1240, D-5205 St. Augustin, 1, F. R. G.] - and D. Ophir, 1984: GRIDPACK: Toward unification of gen .eral grid programming. PDE Software: Modules, Interfaces and Systems. B. Engquist and T. Smedsaas, Eds., Elsevier, 269-288. , J. E. Dendy, Jr. and H. Ruppel, 1980: The multigrid method for semi-implicit hydrodynamics codes. J. Comput. Phys., 34, 348-370. , S. McCormick and J. Ruge, 1983i Multigrid methods for dif ferential eigenproblems. SIAM J. Sci. Stat. Comput., 4, 244 260. , S. R. Fulton and G. D. Taylor, 1985: Improved spectral mul tigrid methods for periodic elliptic problems. J. Comput. Phys., 58, 96-1 l 2.Buzbee, B. L., G. H. Golub and C. W. Nielson, 1970: On direct methods for solving Poisson's equations. SIAM J. Numer. Anal., 7, 627-656.Caughey, D. A., 1983: Multi-grid calculation of three-dimensional transonic potential flows. Appl. Math. Comp., 13, 241-260.Deconinck, H., and C. Hirsch, 1982: A multigrid finite element method for the transonic potential equation. Multigrid Methods. W. Hackbusch and U. Trottenberg, Eds., Lecture Notes in Mathematics, Vol. 960; Springer-Verlag, 387-409.Dendy, J. E., Jr., 1982: Black box multigrid. J. Comput. Phys., 48, 366-386.--, 1983: Black box multigrid for nonsymmetric problems. Aj~pl. Math. Comp., 13, 261-283.Fedorenko, R. P., 1962: A relaxation method for solving elliptic dif ference equations. U.S.S.R. Comput. Math. Math. Phys., 1, 1092-1096.,1964: The speed of convergence of an iterative process. U.S.S.R.Comput. Math. Math. Phys., 4, 227-235.Foerster, H., and K. Witsch, 1982: Multigrid software for the solution of elliptic problems on rectangular domains: MG00 (Release 1). Multigrid Methods. W. Hackbusch and U. Trottenberg, Eds., Lecture Notes in Mathematics, Vol. 960, Springer-Verlag, 427 460.Gary, J., S. Mccormick and R. Sweet, 1983: Successive overrelax ation, multigrid, and preconditioned conjugate gradients algo rithms for solving a diffusion problem on a vector computer. Appl. Math. Comp., 13, 285-309.Ghia, U., K. N. Ghia and C. T. Shin, 1982: High-Re solutions for incompressible flow using the Navier-Stokes equations and a multigrid method. J. Comput. Phys., 48, 387-411.Hackbusch, W., 1979: On the computation of approximate eigen values and eigenfunctions of elliptic operators by means of a multi-grid method. SIAMJ. Numer. Anal., 16, 201-215.Haltiner, G. J., and R. T. Williams, 1980: Numerical Prediction and Dynamic Meteorology, second edition. J. Wiley and Sons, 477 PP.Hemker, P. W., 1982: Mixed defect correction iteration for the ac curate solution of the convection diffusion equation. Multigrid Methods. W. Hackbusch and U. Trottenberg Eds., Lecture Notes in Mathematics, Vol. 960, Springer-Verlag, 485-501. -, R. Kettler, P. Wesseling and P. M. de Zeeuw, 1983: Multigrid methods: Development of Past solvers. Appl. Math. Comp., 13, 311-326.Hoskins, B. J., 1975: The geostrophic momentum approximation and the semi-geostrophic equations. J. Atmos. Sci., 32, 233 242.---, and I. Draghici, 1977: The forcing of ageostrophic motion according to the semi-geostrophic equations and in an isentropic coordinate model. J. Atmos. Sci., 34, 1859-1867.Jameson, A., 1983: Solution' of the Euler equations for two dimen sional transonic flow by multigrid method. Appl. Math. Comp., 13, 327-355.Jespersen, D., i 983: Design and implementation ofa multigrid code for the Euler equations. Appl. Math. Comp., 13, 357-374.Johnson, G. M., 1983: Multiple-grid convergence acceleration of vis cous and inviscid flow computations. Appl. Math. Comp., 13, 375-398.Kettler, R., 1982: Analysis and comparison of relaxation schemes in robust multigrid and preconditioned conjugate gradient methods. Multigrid Methods. W. Hackbusch and U. Trottenberg, Eds., Lecture Notes in Mathematics, Vol. 960, Spdnger-Verlag, 502 '534.Lee, H. N., and R. E. Meyers, 1980: On time dependent multi-grid numerical technique. Comp. Math. Appl., 6, 61-65.Lindzen, R. S., and H.-L. Kuo, 1969: A reliable method for thenumerical integration of a large class of ordinary and partialdifferential equations. Mon. Wea. Rev., 97, 732-734.McCarthy, D. R., and R. C. Swanson, 1983: Embedded mesh mul tigrid treatment of two-dimensional transonic flows. Appl. Math. Comp., 13, 399-417.Meis, Th., H. Lehmann and H. Michael, 1982: Application of the multigfid method to a nonlinear indefinite problem. Multigrid Methods. W. Hackbusch and U. Trottenberg Eds., Lecture Notes in Mathematics, Vol. 960, Springer-Verlag, 545-557.Mittlemann, H. D., and H. Weber, 1985: Multi-grid solution of bi furcation problems. SIAM J. Sci. Stat. Comp., 6, 49-60.Nicolaides, R. A., 1977: On the l2 convergence of an algorithm for solving finite element equations. Math. CoreR., 31, 892-906.Ogura, Y., and N. A. Phillips, 1962: Scale analysis of deep and shallow convection in the atmosphere. J. Atmos. Sci., 19, 173-179.Rice, J. R., 1984: Building elliptic problem solvers with ELLPACK. Elliptic Problem Solvers II.. G. Birkhoff and A. Schoenstadt, Eds., Academic Press, 3-22.Schaffer, S., 1984: Higher order multi-grid methods. Math. Comp., 43, 89-115.Schubert, W. H., and J. J. Hack, 1983: Transformed Eliassen balanced vortex model. J. Atmos. Sci., 40, 1571-1583.MAY 1986 FULTON, CIESIELSKI AND SCHUBERT 959Stoer, J., and R. Bulirsch, 1980: Introduction to NumericalAnalysis.Springer-Verlag, 609 pp.SWeett, C. L., T. A. Zang and M. Y. Hussaini, 1985: Spectral multigrid methods with applications to transonic potential flow. J. Comput. Phys., 57, 43-76.Stfiben, K., 1983: Algebraic multighd (AMG): Experiences and com parisons. Appl. Math. Comp., 13, 419-452. , and U. Trottenberg, 1982: Multigrid methods: Fundamental algorithms, model problem analysis and applications. Multigrid Methods. W. Hackbusch and U. Trottenberg, Eds., Lecture Notes in Mathematics, Vol. 960, Springer-Verlag, 1-176. ... , U. Trottenberg and K. Witsch, 1984: Software development based on multigrid techniques. PDE Software: Modules, Inter faces and Systems. B. Engquist and T. Smedsaas, Eds., Elsevier, 241-263.Swarztrauber, P. N., 1977: The methods of cyclic reduction, Fourier analysis and the FACR algorithm for the discrete solution of Poisson's equation on a rectangle. SIAM Review, 19, 490-501.Sweet, R. A., 1977: A cyclic reduction algorithm for solving block tridiagonal systems of arbitrary dimension. SIAM J. Numer. Anal., 14, 706-720.Thorpe, A. J., 1985: Diagnosis of balanced vortex structure using potential vorticity. J. Atmos. Sci., 42, 397-406.Van Rosendale, j., 1983: Algorithms and data structures for adaptive multigrid elliptic solvers. Appl. Math. Comp., 13, 453-470.Wesseling, P., 1982: A robust and efficient multigrid method. Mu/tigridMethods. W. Hackbusch and U. Trottenberg Fads., LectureNotes in Mathematics, Vol. 960, Springer-Vefiag, 614-630.Zang T. A., Y. S. Wong and M. Y. Hussaini, 1982: Spectral multigrid methods for elliptic equafons. J. Comput. Phys., 48, 485-501.---, Y. S. Wong and M. Y. Hussaini, 1984: Spectral multigrid methods for elliptic equations IL J. Comput. Phys., 54, 489 507.