Climate data access, data analysis, data display, and understanding of the climate system have evolved significantly over the past 40 years.
This paper briefly reviews the evolution in climate data access, data analysis and display, and our understanding of the climate system as reflected in a series of climate diagnostics and prediction workshops, initiated in 1976, and held annually for the past 40 years. This review is by no means comprehensive and reflects our individual experiences and interests.
BACKGROUND.
In 1975, the National Climate Program was initiated in response to several stimuli including the spectacular financial coup by the Soviet Union in securing the bulk of the world’s wheat futures at bargain prices in the face of massive crop shortfalls in the world’s principal grain-growing areas. These shortfalls were thought to have been a result of unusual climate conditions in many of these areas. In addition to seasonal and year-to-year climate variations, there was also some concern about long-term changes, including both warming and cooling. Suffice it to say, in response to these concerns the U.S. government decided that some resources needed to be focused on climate issues (Reeves and Gemmil 2004).climatology as presently practiced is primarily a statistical study without the basis of physical understanding, which is essential for progress…there has been a woeful tendency to the use of the bones of bare statistics and mean values without the flesh of physical understanding. (Durst 1951)
One response to this need was the formation of a small working group within the National Oceanic and Atmospheric Administration (NOAA) that had the initial task of determining whether the state of climate science in 1975 was any different than what was stated in the Compendium 25 years earlier. The initial NOAA Climate Diagnostics Workshop held in Washington, D.C. (actually Camp Springs, Maryland), 4–7 November 1976, was one early attempt to address this question. The workshop was to take stock of the then-current state of climate science and to gauge how much (if at all) progress had been made from the “bones of bare statistics” toward the “flesh of physical understanding” in the quarter of a century since the Compendium was published.
THE EARLY WORKSHOPS.
Proceedings were published to document the activities of this first workshop. That the workshops were primarily a federal government activity was reinforced by its title “Proceedings of the NOAA Climate Diagnostics Workshop.” This scope is confirmed by a look at the attendance list. Of the 62 participants, 50 were federal government employees, mostly from NOAA; 9 were from the academic community; and 3 came from the private sector.
In the introduction to the proceedings of the first workshop Ed Epstein stated that the workshop aimed to include discussions of the “current awareness of the state of the climate” as well as in-depth studies of particular events, statistical diagnostics calculations, synoptic approaches, and observational studies, with the purpose to “To share and dispute views in a forum that will be listened to.” The use of numerical climate models for climate analysis was not explicitly mentioned except for a prescient statement in the preface of the first workshop proceedings by Bill Sprigg, who wrote “Hope for Numerical Models, as primitive as they may be at this time, will help to develop climate diagnostics.”
The proceedings of the 1976 workshop contained 23 papers. Some of the papers concentrated on topics that did not appear in subsequent workshops. Titles of other papers would not sound out of place on a contemporary agenda. For example, these included “Estimates of the global change in temperature, surface to 100 mb” by J. K. Angell and J. Korshover; “Survey of major Northern Hemisphere seasonal anomalies” by A. J. Wagner (Wagner 1976); “An improved approach for following and predicting equatorial Pacific changes and El Nino” by W. H. Quinn; and “Circulation over the tropics and ocean warming in the eastern Pacific during 1976” by A. F. Kruger (Kruger 1976).
There were also papers addressing recent droughts in Australia, Europe, the Great Plains, and California. Some figures from the proceedings of this first workshop illustrate the state of climate analysis at that time. Few historical analyzed upper-air data were widely available in the mid-1970s. The historical 700-mb data, for example Fig. 1, existed in large part, because the Long Range Prediction group in the National Weather Service (NWS), situated at the National Meteorological Center (NMC), went to great pains to have these data archived onto a standard grid (Barnston and Livezey 1987). These upper-air data formed the basis for monthly and (experimental) seasonal climate predictions based on “teleconnection” statistics. The analysis domain was restricted to the Northern Hemisphere north of about 20°N.
The 700-mb heights and anomalies from the NMC twice-daily analysis for fall of 1975 (Wagner 1976).
Citation: Bulletin of the American Meteorological Society 98, 3; 10.1175/BAMS-D-16-0012.1
Real-time sea surface temperature (SST) analyses were also in their early stages of development, as illustrated in the workshop paper by Kruger (1976) (Fig. 2). This paper and two or three others in the proceedings of this first workshop demonstrate that there was awareness of the El Niño–Southern Oscillation (ENSO) even though ENSO had not yet become a central research topic. Note in Fig. 2 that the sea surface temperature time series is hand drawn, goes back to only to 1970, and is for a location just off the west coast of Ecuador (5°S, 85°W). It would fall in the current Niño-1+2 area (see, e.g., the Climate Prediction Center website: www.cpc.ncep.noaa.gov). The sources of the data are not identified but are likely to have been from the “Fishing Information Bulletin” published by the National Marine Fisheries Service’s (NMFS) Southwest Fisheries Center in La Jolla, California. The summer and fall 1976 data may have come from another source, possibly R. M. Laurs (NMFS) or Bill Quinn (Oregon State University). The atmospheric circulation data (not shown) were from the NMC tropical strip analysis.
Circulation over the tropics and ocean warming over the eastern Pacific during 1976 (Kruger 1976, his Fig. 1).
Citation: Bulletin of the American Meteorological Society 98, 3; 10.1175/BAMS-D-16-0012.1
Satellite data, as opposed to satellite imagery, were in the early days of exploitation for use in weather and climate analysis. A paper in the proceedings, presented by Jay Winston (Winston 1976), later to become the first director of NOAA’s Climate Analysis Center, focused on outgoing longwave radiation (OLR) (Fig. 3). Since satellite observations of OLR were relatively new, there was no “climatology.” In lieu of anomalies from a climatology, Winston and others in the early years of climate diagnostics used year-to-year seasonal differences of OLR and other variables to compare one year to another. In his paper, Winston noted that the eastward shift in negative OLR “anomalies” was consistent with warming in the equatorial Pacific.
OLR variations for 1975–76 compared to 1974–75 (Winston 1976). Negative OLR anomalies are shaded.
Citation: Bulletin of the American Meteorological Society 98, 3; 10.1175/BAMS-D-16-0012.1
These examples and others that could have been cited from the first workshop provide a means for summarizing the state of climate data access, data analysis and display, and our understanding of the climate system in the mid-1970s, as given below.
Climate data access.
Many of the participants at this first Climate Diagnostics Workshop went through great efforts to present up-to-date climate information. However, many datasets cited in the presentations were from sources that were not available to all investigators and often were held by individual researchers or their institutions. There were almost no satellite or numerical model–based climate data. There were almost no Southern Hemisphere or global ocean data.
Data analysis and display.
Climate anomalies were based on short periods of record often with sparse data. Some analyses were limited to year-to-year differences. The presentations relied on transparencies displayed by use of overhead projectors and often drafted from hand analyses.
Understanding of the climate system.
There was some awareness of ENSO but not of its global impacts. There was no mention of the Madden–Julian oscillation (MJO) or the North Atlantic Oscillation (NAO), the Pacific–North American (PNA) teleconnection, or other circulation patterns. Based on the first workshop, one might argue that climate science had made only modest progress in starting to move away from the bones of bare statistics.
This workshop, though dominated by participants from the federal government, recognized the need for involvement by the wider community, most notably the academic and research sectors. This realization led to seeking cosponsorship of subsequent workshops by universities or research organizations, a custom that continues to this day.
The first workshop was an informal affair with no official registration fees, no banquet, and thus, no after-dinner speakers. The organizers often asked for voluntary donations to pay for coffee and doughnuts during the breaks. This informality continued until the 13th workshop, in 1988. By the 13th workshop the number of attendees had increased to the point that local cosponsors found it difficult to find free venues large enough to accommodate the meeting. This led to the introduction of registration fees and banquets (as well as after-dinner speakers).
The second workshop was held at Scripps Institution of Oceanography, University of California, San Diego, and the third at the Cooperative Institute of Marine and Atmospheric Studies (CIMAS) at the University of Miami. Both were organized by the National Climate Program office. By the fourth workshop, the Climate Analysis Center (CAC) had been established and part of CAC’s responsibilities was to engage with the wider climate research community. The continuation of this series of meetings was one way that this could be facilitated. At the fourth workshop, held at the University of Wisconsin in Madison in October 1979, the attendance had grown to 83 people, with 49 from the academic/research community and 3 from the private sector (59% academic, 3.6% private sector). The fifth workshop was held at the University of Washington and the sixth at Lamont-Doherty Earth Observatory (LDEO) of Columbia University. It was during the seventh workshop (October of 1982) held at the National Center for Atmospheric Research (NCAR) in Boulder, Colorado, that the great 1982/83 ENSO episode was underway, something of which most of the attendees were not fully aware. Because of this uncertainty, this meeting was considered by many to be the most “workshop like” up until that time. While many participants had strongly held opinions about the state of ENSO, those opinions did not, in general, surface during the formal presentations with one exception. One of the leading experts of the day gave an El Niño forecast stating “It is predicted that no moderate or strong El Niño event will occur in 1983 and it is unlikely that such a potential can develop before 1984.” This statement reached legendary status within the climate community. Within a week of the conclusion of this workshop the climate community discovered not only that there was an ENSO in progress but also that it was, up until that time, the strongest ENSO episode of the twentieth century.
Why did the community in general miss the onset of the 1982/83 ENSO? One reason is that there was another large geophysical event earlier in 1982: the eruption of the El Chichón volcano in Mexico that spring. That eruption was the subject of many of the discussions at the workshop, with a number of workshop presentations describing the effects and potential impacts of the El Chichón eruption on climate. In fact, there was a special El Chichón workshop session consisting of nine papers. Perhaps the thinking at the time may be best summarized in the workshop proceedings in a paper by Jay Winston, “the broad scale radiative impacts of the volcanic eruption will be difficult to separate from the [unusual natural] climate anomaly of the subtropics this spring.” It turned out that the contamination of the satellite-based analyses by the volcanic aerosols produced SST estimates that were too low. At the same time the in situ measurements of record high SST in the eastern Pacific were discarded by automated quality control algorithms and unrealistic outliers. Thus, the observational systems of that day were not up to the task of identifying the record high SST anomalies in the equatorial Pacific without closer examination.
Even though many of the participants of the seventh workshop may have failed to recognize the ongoing ENSO, in many ways this meeting marks some significant progress in climate analysis.
Climate data access.
Despite considerable efforts at the CAC and in the climate community at large, access to “real time” climate data complemented by historical records of sufficient length, required to form meaningful climate anomalies, continued to be less than ideal. Nonetheless, one important step toward providing better data was the implementation of the Climate Diagnostics Data Base (CDDB; Fig. 4), which made some of the CDDB data available to the climate research community. The atmospheric component of the CDDB was the global database of half-monthly statistics of wind components, temperatures, geopotential heights, and moisture at a range of pressure levels from near the surface to the lower stratosphere. A subset of CDDB data was made available to the wider research community. This subset consisted of the u and υ wind components at 850 and 250 mb, together with OLR and the Reynolds (1988) blended SST. This data subset was among the most frequently requested from the CAC and was used by a large segment of the research community. It was very often featured in subsequent workshops until the advent of the NCEP–NCAR reanalysis (Kalnay et al. 1996). The complete CDDB was not, however, ready for prime time in 1982. In subsequent years CDDB data were archived at half-monthly intervals and sent to the National Climatic Data Center (NCDC), now the National Centers for Environmental Information (NCEI), in Asheville, North Carolina, for distribution. In addition, a larger subset of the CDDB was extracted and made available to the research community by the CAC upon request.
Schematic showing the components of the CDDB. Note the evolution in the institutional structure. NMC is now the NCEP, and the National Environmental Satellite Service (NESS) and the Environmental Data and Information Service (EDIS) were merged into NESDIS.
Citation: Bulletin of the American Meteorological Society 98, 3; 10.1175/BAMS-D-16-0012.1
Data analysis and display.
Analysis techniques continued to evolve. In particular, there was an increased use of empirical orthogonal functions (EOFs) and related analyses. A comment at the seventh workshop from one of the pioneers of the U.S. Weather Bureau’s long-range prediction efforts, “Why all of a sudden all of these papers with EOFs?” Most climate analyses continued to be performed on large mainframe computers. Personal computers, workstations, and desktop plotters had not yet come into common usage.
Understanding of the climate system.
By 1982 the climate community was familiar, to some degree, with El Niño and its relationship to the Southern Oscillation. The paper by Rasmussen and Carpenter (1982) had just been published, describing the “canonical” relationships between the El Niño warming in the eastern Pacific Ocean and Walker’s (1924) Southern Oscillation. The term, ENSO, which later came to be used for the El Niño–Southern Oscillation, however, was not used at the workshop nor in the classic paper. Much of the terminology that is now familiar, not just to climate scientists but also to many outside the field, including the terms ENSO and La Niña, came into common usage shortly afterward. The 1982/83 warm episode, or El Niño, dramatically changed the climate science community’s perceptions and understanding of ENSO and inspired an enormous body of work, still evolving, that describes its global impacts and its theoretical basis. The development of empirical and model-based ENSO predictions as tools to improve operational seasonal forecasts began after the 1982/83 episode.
GROWTH YEARS (8TH–20TH WORKSHOPS).
The eighth workshop was held at the Canadian Climate Center in Downsview, Ontario, the only workshop held outside of the United States. The ninth was held at the Oregon State University in Corvallis, completing the round of workshops held at institutions with an early interest in contemporary climate analysis. The 10th workshop, held at the University of Maryland, was cosponsored by the World Meteorological Organization (WMO) and had the unwieldy title “First WMO Workshop on the Diagnosis and Prediction of Monthly and Seasonal Atmospheric Variations over the Globe (Combined with NOAA’s 10th Climate Diagnostics Workshop).” This workshop was one indication of the extent to which climate analysis and research was moving from a backwater of atmospheric and oceanic endeavor to one of the cutting-edge topics. Nonetheless, the phrase “atmospheric variations” in the lengthy title suggests that role of the oceans and land surface in the climate variations had not yet been completely assimilated.
The list of the first 20 workshops can be found in Table 1. For the most part, the workshops continued to be cosponsored by universities, with the exception of the 13th, which was cohosted by Atmospheric and Environmental Research (AER), a private company based in Cambridge, Massachusetts. The Loma Prieta earthquake that shook San Francisco, California, added excitement to the 14th workshop in La Jolla, as did the after-dinner talk on another kind of geophysical chaos by Ed Lorenz. The size of the workshops was continuing to grow from 96 attendees at the 11th workshop at the University of Illinois to 139 at the 15th at the NCDC, with 47 and 59 attendees from the university community, respectively. The 16th–20th workshops continued to reflect rapid progress in climate diagnostics that gave rise to an increasing number of papers describing the far-ranging influence of ENSO and more presentations examining climate variability on other time scales. By the 20th workshop in 1995, held at the NOAA/Pacific Marine Environmental Laboratory in Seattle, Washington, it had become common for the workshop to contain a number of presentations that included diagnostics of the ocean as a part of a coupled climate system. By the 20th workshop substantial progress had been made in all aspects of climate research.
Climate diagnostics workshops (1976–95).
Climate data access.
The CDDB, the Climate Anomaly Monitoring System (CAMS), the Reynolds (1988) blended SST dataset, the Comprehensive Ocean–Atmosphere Dataset (COADS), satellite-derived near-global precipitation estimates, the normalized difference vegetation index (NDVI), and several other climate datasets had become more widely available. “Global” data began to include not only the tropics, but also the Southern Hemisphere. The Internet was not yet fully developed so that data access and exchange was still somewhat cumbersome, often involving mailing 7- or 9-track magnetic tapes.
Data analysis and display.
EOF analysis and its variants were now standard components of a climate researcher’s tool kit. With the availability of powerful (for the time) computer workstations from Apollo and Sun, computation and display of complex analyses could be carried out on the researcher’s desktop. Graphical display software such as NCAR Graphics (http://ngwww.ucar.edu/index.html) and the Grid Analysis and Display System (GrADS; Doty and Kinter 1995) began to be developed and used, and coupled with desktop printing and publishing ended the era of handcrafted “Sharpie” color graphics (Fig. 5). The CDDB was a significant step forward in attempts to monitor and analyze the state of the atmosphere in real time based on numerical model analyses. However, changes in operational forecast models and data assimilation systems, essential to improving weather forecasts, limited the ability of the CDDB data to distinguish changes due to real climate variability to a much greater degree than had been anticipated. This clearly identified the need for a “stable” Climate Data Assimilation System (CDAS) to analyze the climate system and for a complementary reanalysis of historical observations to permit the calculation of consistent anomaly fields. The 19th workshop in 1994 marked the end to the CDDB era for routine monitoring and analysis with the introduction of the NCEP–NCAR reanalysis (Kalnay et al. 1996); see Fig. 6.
An example of a hand-colored overhead transparency typical of the data displays during the workshops of the 1980s and 1990s. Computer-generated presentations (e.g., PowerPoint) began to take over in the mid- to late 1990s.
Citation: Bulletin of the American Meteorological Society 98, 3; 10.1175/BAMS-D-16-0012.1
Cover of the 19th workshop proceedings illustrating the difference between climate analyses derived from the operational climate model (dotted line) and from the NCEP–NCAR reanalysis (solid line). (top) Mean virtual 1,000-mb temperature. (bottom) Mean 1,000-mb wind speed, in the tropics. (From the paper by M. Chelliah in the proceedings.)
Citation: Bulletin of the American Meteorological Society 98, 3; 10.1175/BAMS-D-16-0012.1
Understanding of the climate system.
ENSO and its teleconnections became widely studied, better understood, and the default “usual suspect” for every observed climate anomaly whether merited by strong evidence or not. The natural reaction of climate scientists to this state of affairs led to growing interest in variability beyond ENSO, including low-frequency variability and blocking, the MJO, as well as other patterns of interannual climate variability.
DIAGNOSTICS AND PREDICTION (21ST–40TH WORKSHOPS).
The 20th workshop, held in 1995, was the last in this series of meetings to be called a climate diagnostics workshop. By 1995, NMC had become the National Centers for Environmental Prediction (NCEP), the Climate Analysis Center (CAC) had become the Climate Prediction Center (CPC), and climate diagnostics had become a widely practiced activity within the climate research community and the principal focus of the NOAA/Climate Diagnostics Center in Boulder. Throughout its 20-year history the climate diagnostic workshops included sessions on climate prediction. Often, however, prediction was addressed in a half-day session at the end of the workshop and it was common in the early days for most of the attendees to be leaving while these sessions were being held.
Increased understanding of the climate system along with the introduction of coupled numerical models into the mix suggested that a stronger focus on climate prediction was needed. Thus, the focus of the climate diagnostics workshop was expanded to include prediction and the initiation of the climate diagnostics and prediction workshops. These workshops were viewed as an expansion and continuation of the previous 20 meetings.
The 21st Climate Diagnostics and Prediction Workshop (1996), at the University of Alabama in Huntsville, was the first of the expanded workshops. A listing of the locations of the 21st–40th Climate Diagnostics and Prediction Workshops can be found in Table 2. The change in the workshop was heralded by the figure on the cover of the proceedings showing verification statistics; the addition of three sessions on prediction, including one on the first day; and one session on model diagnostics. Subsequent workshops included an increased number of papers on prediction and predictability as well as an increased dependence on model-based analyses and experiments.
Climate diagnostics and prediction workshops (1996–2015).
The workshops continued to evolve. One indication of this evolution was the 25th workshop (2000), cosponsored by the International Research Institute for Climate and Society (IRI) at the LDEO campus of Columbia University, with its strong focus on the societal impacts of climate variability. The content of workshop papers started to evolve, so that, rather than separate sessions on analysis and prediction, workshop sessions now often include a mix of analysis, prediction, and climate impacts papers focused on particular topics such as the ENSO or the MJO.
CLOSING COMMENTS.
This 40-year series of climate diagnostics and prediction workshops make it clear that climate science has evolved enormously to include data, analyses, and topics barely imagined at the first workshop.
Climate data access.
The World Wide Web and advances in the ability to calculate, store, and exchange data have revolutionized climate science. The view of climate data, traditionally thought of as long records of observations taken at individual locations, has largely given way to global gridded datasets based on observations, numerical model analyses, and satellite-derived analyses. Early global gridded climate data were restricted to monthly or seasonal values with typical spatial dimensions of 2.5° latitude by 2.5° longitude or larger. Some numerical model-based and satellite-based datasets for climate studies are now available at 3-hourly or less temporal resolutions and less than 1° latitude–longitude spatial scales. Much of these data can be accessed from data centers with a few computer keystrokes. Challenges to analysts now include how to select data appropriate to their study and how to relate data in a model, or analyzed grid, to the climate data at a particular location.
Data analysis and display.
As with the access to data, the analysis of data has witnessed revolutionary changes associated with the advances in computer power and resources available on Internet. Most climate analysts have access to standard analysis packages or can find them on the Internet. Current workshop talks often include animated color graphics.
Understanding of the climate system.
The climate system is now understood to include interactions among the ocean, atmosphere, land surface, and cryosphere. Several patterns of climate variability have been identified and are the subject of active research. It is clear that climate science has moved beyond the “bones of bare statistics” but there are still a number of bones to pick. Our ability to observe, to describe, and to model climate variability and change has witnessed significant advances, as reflected in the evolution of the workshop contents over the past 40 years. Nonetheless, many climate phenomena, while well described, are not fully understood. It remains a challenge for climate scientists and future workshops to continue making progress toward the “basis of physical understanding.”
ACKNOWLEDGMENTS
This paper is based on a presentation given at the 40th Climate Diagnostics and Prediction Workshop. Thanks to the CPC for suggesting this topic and also to Bob Reeves and Rick Rosen for discussions and information about the early workshops. We also thank an anonymous reviewer for several insightful comments. A special thanks from CFR to the staff of the NOAA Library in Silver Spring, Maryland, for help in accessing the workshop proceedings and other historical material.
REFERENCES
Barnston, A. G., and R. E. Livezey, 1987: Classification, seasonality and persistence of low-frequency atmospheric circulation patterns. Mon. Wea. Rev., 115, 1083–1126, doi:10.1175/1520-0493(1987)115<1083:CSAPOL>2.0.CO;2.
Doty, B., and J. L. Kinter III, 1995: Geophysical data analysis and visualization using GrADS. Visualization Techniques in Space and Atmospheric Sciences, E. P. Szuszczewicz and J. H. Bredekamp, Eds., NASA, 209–219.
Durst, C. S., 1951: Climatology—A synthesis of weather. Compendium of Meteorology, T. F. Malone, Ed., Amer. Meteor. Soc., 967–975.
Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-Year Reanalysis Project. Bull. Amer. Meteor. Soc., 77, 437–471, doi:10.1175/1520-0477(1996)077<0437:TNYRP>2.0.CO;2.
Kruger, A., 1976: Circulation over the tropics and ocean warming over the eastern Pacific during 1976. Proc. NOAA Climate Diagnostics Workshop, Washington, DC, NOAA, 11-1–11-8. [NTIS PB-264993.]
Malone, T. F., 1951: Compendium of Meteorology. Amer. Meteor. Soc., 1334 pp.
Reeves, R. W ., and D. deJ. Gemmil, Eds., 2004: Reflections on 25 Years of Analysis, Diagnosis, and Prediction: 1979–2004. Climate Prediction Center, 106 pp. [Available online at www.cpc.ncep.noaa.gov/products/outreach/anniversary.pdf.]
Reynolds, R. W., 1988: A real-time global sea surface temperature analysis. J. Climate, 1, 75–87, doi:10.1175/1520-0442(1988)001<0075:ARTGSS>2.0.CO;2.
Wagner, A. J., 1976: 700mb heights and anomalies from the NMC twice-daily analysis for fall 1975. Proc. NOAA Climate Diagnostics Workshop, Washington, D.C., NOAA. [NTIS PB-264993.]
Walker, G. T., 1924: Correlation of the seasonal variations in weather IX: A further study of world weather. Mem. Indian Meteor. Dep., 24, 275–332.
Winston, J. S., 1976: Outgoing longwave radiation variations (1975–1976) compared to (1974– 1975). Proc. NOAA Climate Diagnostics Workshop, Washington, D.C., NOAA. [NTIS PB-264993.]
