Editorial Type:
Article
Article Type:
Research Article

The Computational Complexity and Parallel Scalability of Atmospheric Data Assimilation Algorithms

P. M. Lyster Global Modeling and Assimilation Office, Laboratory for Atmospheres, NASA Goddard Space Flight Center, Greenbelt, Maryland

Search for other papers by P. M. Lyster in
Current site
Google Scholar
PubMed Close
,
J. Guo Global Modeling and Assimilation Office, Laboratory for Atmospheres, NASA Goddard Space Flight Center, Greenbelt, Maryland

Search for other papers by J. Guo in
Current site
Google Scholar
PubMed Close
,
T. Clune Global Modeling and Assimilation Office, Laboratory for Atmospheres, NASA Goddard Space Flight Center, Greenbelt, Maryland

Search for other papers by T. Clune in
Current site
Google Scholar
PubMed Close
, and
J. W. Larson Global Modeling and Assimilation Office, Laboratory for Atmospheres, NASA Goddard Space Flight Center, Greenbelt, Maryland

Search for other papers by J. W. Larson in
Current site
Google Scholar
PubMed Close
Print Publication:
01 Nov 2004
DOI:
https://doi.org/10.1175/JTECH1636.1
Page(s):
1689–1700
Restricted access

Abstract

This paper quantifies the computational complexity and parallel scalability of two algorithms for four-dimensional data assimilation (4DDA) at NASA's Global Modeling and Assimilation Office (GMAO). The first, the Goddard Earth Observing System Data Assimilation System (GEOS DAS), uses an atmospheric general circulation model (GCM) and an observation-space-based analysis system, the Physical-Space Statistical Analysis System (PSAS). GEOS DAS is very similar to global meteorological weather forecasting data assimilation systems but is used at NASA for climate research. The second, the Kalman filter, uses a more consistent algorithm to determine the forecast error covariance matrix than does GEOS DAS. For atmospheric assimilation, the gridded dynamical fields typically have more than 106 variables; therefore, the full error covariance matrix may be in excess of a teraword. For the Kalman filter this problem will require petaflop s−1 computing to achieve effective throughput for scientific research.

Additional affiliation: Earth System Science Interdisciplinary Center, and Department of Meteorology, University of Maryland, College Park, College Park, Maryland

Additional affiliation: Science Applications International Corporation/General Sciences Operation, Beltsville, Maryland

Additional affiliation: Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, Illinois

Corresponding author address: Dr. Peter M. Lyster, National Institutes of Health, Bethesda, MD 20892. Email: lysterpe@nigms.nih.gov

Abstract

This paper quantifies the computational complexity and parallel scalability of two algorithms for four-dimensional data assimilation (4DDA) at NASA's Global Modeling and Assimilation Office (GMAO). The first, the Goddard Earth Observing System Data Assimilation System (GEOS DAS), uses an atmospheric general circulation model (GCM) and an observation-space-based analysis system, the Physical-Space Statistical Analysis System (PSAS). GEOS DAS is very similar to global meteorological weather forecasting data assimilation systems but is used at NASA for climate research. The second, the Kalman filter, uses a more consistent algorithm to determine the forecast error covariance matrix than does GEOS DAS. For atmospheric assimilation, the gridded dynamical fields typically have more than 106 variables; therefore, the full error covariance matrix may be in excess of a teraword. For the Kalman filter this problem will require petaflop s−1 computing to achieve effective throughput for scientific research.

Additional affiliation: Earth System Science Interdisciplinary Center, and Department of Meteorology, University of Maryland, College Park, College Park, Maryland

Additional affiliation: Science Applications International Corporation/General Sciences Operation, Beltsville, Maryland

Additional affiliation: Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, Illinois

Corresponding author address: Dr. Peter M. Lyster, National Institutes of Health, Bethesda, MD 20892. Email: lysterpe@nigms.nih.gov

Save
Cover Journal of Atmospheric and Oceanic Technology
Journal of Atmospheric and Oceanic Technology

  • Amdahl, G. M., 1967: Validity of the single-processor approach to achieving large scale computing capabilities. Proc. AFIPS Spring Joint Computer Conf., Atlantic City, NJ, AFIPS, 483– 485.

    • Search Google Scholar
    • Export Citation

  • Andersson, E., and Coauthors, 1998: The ECMWF implementation of three dimensional variational assimilation (3D-Var). Part III: Experimental Results. Quart J. Roy. Meteor. Soc, 124 , 1831–1860.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Atlas, R., 1997: Atmospheric observations and experiments to assess their usefulness in data assimilation. J. Meteor. Soc. Japan, 75 , 111–130.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Atlas, R., Hoffman R. N. , Bloom S. C. , Jusem J. C. , and Ardizzone J. , 1996: A multiyear global surface wind velocity dataset using SSM/I wind observations. Bull. Amer. Meteor. Soc, 77 , 869–882.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Bloom, S. C., Takacs L. L. , da Silva A. M. , and Ledvina D. , 1996: Data assimilation using incremental analysis updates. Mon. Wea. Rev, 124 , 1256–1271.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Brackbill, J. U., and Cohen B. I. , Eds.,. 1985: Multiple Time Scales. Academic Press, 442 pp.

  • Cohn, S. E., 1997: An introduction to estimation theory. J. Meteor. Soc. Japan, 75 , 257–288.

  • Cohn, S. E., da Silva A. , Guo J. , Sienkiewicz M. , and Lamich D. , 1998: Assessing the effects of data selection with the DAO Physical-space Statistical Analysis System. Mon. Wea. Rev, 126 , 2913–2926.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Courtier, P., and Coauthors, 1998: The ECMWF implementation of three dimensional variational assimilation (3D-Var). Part I: Formulation. Quart. J. Roy. Meteor. Soc, 124 , 1783–1808.

    • Search Google Scholar
    • Export Citation

  • Daley, R., 1991: Atmospheric Data Analysis. Cambridge University Press, 457 pp.

  • da Silva, A., and Guo J. , 1996: Documentation of the Physical-Space Statistical Analysis System (PSAS) part I: The conjugate gradient solver version, PSAS-1.00. NASA Data Assimilation Office Note 96-02, 66 pp. [Available online at http://gmao.gsfc.nasa.gov/;.].

    • Search Google Scholar
    • Export Citation

  • Ding, H., and Ferraro R. , 1995: A general purpose parallel sparse matrix solver package. Proc. Ninth Int. Parallel Processing Symp., Santa Barbara, CA, IEEE, 70.

    • Search Google Scholar
    • Export Citation

  • Foster, I., 1995: Designing and Building Parallel Programs: Concepts and Tools for Parallel Software Engineering. Addison-Wesley, 381 pp.

    • Search Google Scholar
    • Export Citation

  • Gaspari, G., and Cohn S. E. , 1999: Construction of correlation functions in two and three dimensions. Quart. J. Roy. Meteor. Soc, 125 , 723–757.

  • Gibson, J. K., Kållberg P. , Uppala S. , Nomura A. , Hernandez A. , and Serrano E. , 1997: ERA Description. Vol. 1. ECMWF Re-Analysis Project Report Series, ECMWF, 71 pp.

    • Search Google Scholar
    • Export Citation

  • GMAO, cited 2000: Algorithm theoretical basis document version 2.01. Global Modeling and Assimilation Office, NASA Goddard Space Flight Center. [Available online at http://gmao.gsfc.nasa.gov/;.].

    • Search Google Scholar
    • Export Citation

  • Golub, G. H., and van Loan C. F. , 1989: Matrix Computations. 2d ed. The Johns Hopkins University Press, 642 pp.

  • Guo, J., Larson J. W. , Lyster P. M. , and Gaspari G. , cited 1998: Documentation of the Physical-space Statistical Analysis System (PSAS). Part II: The factored-operator error covariance model formulation. NASA Data Assimilation Office Note 98-04, 27 pp. [Available online at http://gmao.gsfc.nasa.gov/;.].

    • Search Google Scholar
    • Export Citation

  • Jazwinski, A. H., 1970: Stochastic Processes and Filtering Theory. Academic Press, 276 pp.

  • Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-Year Reanalysis Project. Bull. Amer. Meteor. Soc, 77 , 437–471.

  • Kistler, R., and Coauthors, 2001: The NCEP–NCAR 50-Year Reanalysis: Monthly means CD-ROM and documentation. Bull. Amer. Meteor. Soc, 82 , 247–268.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Larson, J. W., Guo J. , and Lyster P. M. , 1998: Documentation of the Physical-space Statistical Analysis System (PSAS). Part III: Software implementation. NASA Data Assimilation Office Note 98-05, 189 pp. [Available online at http://gmao.gsfc.nasa.gov/;.].

    • Search Google Scholar
    • Export Citation

  • Lin, S-J., and Rood R. B. , 1996: Multidimensional flux-form semi-lagrangian transport schemes. Mon. Wea. Rev, 124 , 2046–2070.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lyster, P. M., cited 2000: Final report on the NASA HPCC PI project: Four dimensional data assimilation. [Available online at http://ct.gsfc.nasa.gov/lys/hpccfinal.].

    • Search Google Scholar
    • Export Citation

  • Lyster, P. M., Liewer P. C. , Ferraro R. D. , and Decyk V. K. , 1995: Implementation and characterization of three-dimensional particle-in-cell codes on multiple-instruction-multiple-data parallel supercomputers. Comput. Phys, 9 , 420–432.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lyster, P. M., Cohn S. E. , Ménard R. , Chang L-P. , Lin S-J. , and Olsen R. , 1997a: Parallel implementation of a kalman filter for constituent data assimilation. Mon. Wea. Rev, 125 , 1674–1686.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Lyster, P. M., Sawyer W. , and Takacs L. L. , 1997b: Design of the Goddard Earth Observing System (GEOS) Parallel General Circulation Model (GCM). NASA Data Assimilation Office Note 97-13, 17 pp. [Available online at http://gmao.gsfc.nasa.gov/;.].

    • Search Google Scholar
    • Export Citation

  • Lyster, P. M., Clune T. , Guo J. , and Larson J. W. , cited 2000: Performance optimization of the Physical-space Statistical Analysis System (PSAS), 38 pp. [Available online at http://ct.gsfc.nasa.gov/lys/hpccfinal.].

    • Search Google Scholar
    • Export Citation

  • Ménard, R., and Chang L-P. , 2000: Assimilation of stratospheric chemical tracer observations using a Kalman filter. Part II: χ2-validated results and analysis of variance and correlation dynamics. Mon. Wea. Rev, 128 , 2672–2686.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Ménard, R., Cohn S. E. , Chang L-P. , and Lyster P. M. , 2000: Assimilation of stratospheric chemical tracer observations using a Kalman filter. Part I: Formulation. Mon. Wea. Rev, 128 , 2654–2671.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Parrish, D. F., and Derber J. C. , 1992: The National Meteorological Center's spectral statistical-interpolation analysis system. Mon. Wea. Rev, 120 , 1747–1763.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Parrish, D. F., Derber J. C. , Purser R. J. , Wu W-S. , and Pu Z-X. , 1997: The NCEP global analysis system: Recent improvements and future plans. J. Meteor. Soc. Japan, 75 , 359–365.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Rabier, F., Mc Nally A. , Andersson E. , Courtier P. , Undén P. , Eyre J. , Hollingsworth A. , and Bouttier F. , 1998: The ECMWF implementation of three dimensional variational assimilation (3D-Var). Part II: Structure functions. Quart. J. Roy. Meteor. Soc, 124 , 1809–1830.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Riishøjgaard, L. P., 1998: A direct way of specifying flow-dependent background error correlations for meteorological analysis systems. Tellus, 50A , 42–57.

    • Search Google Scholar
    • Export Citation

  • Sawyer, W., and Wang A. , 1999: Benchmark and unit test results of the message-passing GEOS General Circulation Model. NASA Data Assimilation Office Note 99-04, 20 pp. [Available online at http://gmao.gsfc.nasa.gov/;.].

    • Search Google Scholar
    • Export Citation

  • Schubert, S. D., Rood R. B. , and Pfaendtner J. , 1993: An assimilated dataset for earth science applications. Bull. Amer. Meteor. Soc, 74 , 2331–2342.

    • Crossref
    • Search Google Scholar
    • Export Citation

  • Schubert, S. D., and Coauthors, 1995: A multiyear assimilation with GEOS-1 System: Overview and results. Vol. 7. NASA Tech. Memo. 104606, 201 pp. [Available online at http://gmao.gsfc.nasa.gov/;.].

    • Search Google Scholar
    • Export Citation

  • Stobie, J. G., 1996: Data assimilation computing and mass storage requirements for 1998. NASA Data Assimilation Office Note 96-16, 10 pp. [Available online at http://gmao.gsfc.nasa.gov/;.].

    • Search Google Scholar
    • Export Citation

  • Takacs, L. L., Molod A. , and Wang T. , 1994: Documentation of the Goddard Earth Observing System (GEOS) General Circulation Model—Version 1. Vol. 1. NASA Tech. Memo. 104606, NASA Goddard Space Flight Center, Greenbelt, MD, 114 pp. [Available online at http://gmao.gsfc.nasa.gov/;.].

    • Search Google Scholar
    • Export Citation

  • van de Geijn, R. A., Alpatov P. , Baker G. , and Edwards C. , 1997: Using PLAPACK—Parallel Linear Algebra Package. MIT Press, 194 pp.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 304 96 44
PDF Downloads 141 51 1