A Look at the Evolution of Meteorological Satellites: Advancing Capabilities and Meeting User Requirements

Johannes Schmetz EUMETSAT, Darmstadt, Germany

Search for other papers by Johannes Schmetz in
Current site
Google Scholar
PubMed
Close
and
W. Paul Menzel University of Wisconsin–Madison, Madison, Wisconsin

Search for other papers by W. Paul Menzel in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

In this paper, the authors offer their observations from more than 30 years of involvement in the evolution of the space-based meteorological remote sensing systems. Successes and issues from the past are recalled that established meteorological satellites into their current pivotal role. Evolution of imaging and sounding satellite systems from user requirements to affordable realizations is noted; some examples from recent U.S. and European experiences in the area of operational meteorological satellites are presented. The authors discuss the importance of the balanced roles of the three partners in satellite development (government, research, and industry), the need to develop full utilization of new satellite programs quickly during their early life, and a vision for global cooperation early in the planning stages of meteorological satellite missions. The authors offer suggestions that could foster expanded international collaboration on science and applications as well as expedite more satellite observations being pursued in a sustained manner.

Denotes Open Access content.

Corresponding author address: W. Paul Menzel, Space Science and Engineering Center, University of Wisconsin–Madison, 1225 West Dayton St., Madison, WI 53706. E-mail: paul.menzel@ssec.wisc.edu

Abstract

In this paper, the authors offer their observations from more than 30 years of involvement in the evolution of the space-based meteorological remote sensing systems. Successes and issues from the past are recalled that established meteorological satellites into their current pivotal role. Evolution of imaging and sounding satellite systems from user requirements to affordable realizations is noted; some examples from recent U.S. and European experiences in the area of operational meteorological satellites are presented. The authors discuss the importance of the balanced roles of the three partners in satellite development (government, research, and industry), the need to develop full utilization of new satellite programs quickly during their early life, and a vision for global cooperation early in the planning stages of meteorological satellite missions. The authors offer suggestions that could foster expanded international collaboration on science and applications as well as expedite more satellite observations being pursued in a sustained manner.

Denotes Open Access content.

Corresponding author address: W. Paul Menzel, Space Science and Engineering Center, University of Wisconsin–Madison, 1225 West Dayton St., Madison, WI 53706. E-mail: paul.menzel@ssec.wisc.edu

1. Introduction

In today’s world, space-based remote sensing has been established as a means to provide key information on weather and climate of the earth’s system on a global scale. A prime example is found in numerical weather prediction (NWP), where the introduction of new satellite measurements has substantially improved forecasts over recent decades (e.g., Kelly 1997; Pailleux 1997), and meteorological satellite measurements are today a cornerstone for NWP (WMO 2012). The fact that remote sensing measurements are a vital component in the observing system for global monitoring and prediction is being reflected in international partnering toward implementing and planning integrated and coordinated satellite programs. These programs use proven space and sensor technology to develop advanced engineering systems that gather, distribute, archive, and process data at every stage in the information processing chain. One constraint for operational satellite systems is that they need to maintain the continuity of established and necessary services together with affordable innovation. Research missions have a unique and complementary role in probing and demonstrating new capabilities in science and technology.

While the usefulness of meteorological satellite measurements is generally accepted, there are difficulties arising that could hinder and slow down the evolution of the space-based remote sensing systems. A significant problem is the disparity between requirements from user communities and financial commitments from governments. On a positive note, the number of international entities is increasing that are embarking on multibillion dollar weather and meteorological satellite programs; each has the goal of reducing the potentially detrimental impact of weather on life and property as well as enhancing the benefits drawn from more accurate weather (and climate) services. Clearly, this is not only a science- and user-driven activity, because there is also a desire and necessity from industry to prove new capabilities and technologies. We think these elements make possible future scenarios for a space-based observing system that exhibits sustained growth in capability for global information regarding weather and climate. In fact, one could think of a future when the global space-based observing system would be planned and designed with international coordination from the outset, making contributions from individual space agencies truly complementary. This would go beyond the currently established coordination of meteorological satellite measurements. We freely admit that it would be a paradigm shift in many ways and therefore present significant challenges; however, it would make observations, especially the operational, to-be-sustained part of the satellite observing system, more affordable. We also emphasize that there should always be enough room for innovation and free thinking; this is currently well captured in the principal investigator (PI)-driven missions pursued by research space agencies in competitive ways. This needs to be maintained. However, as the research missions are typically “one-off” missions, an immediate question is whether useful research missions need to be continued and, if so, what would be an adequate framework to do so?

In this paper, we summarize our impressions from more than three decades of involvement in the evolution of the imaging and sounding space-based meteorological passive remote sensing systems. We recall some of the successes and issues from the past that established satellites into their current pivotal role.

We note that our perspective is subjective and limited; however, it is from two scientists that transitioned into middle management. We started in a hands-on period contributing to the scientific development of satellite programs, transitioned to guiding the technical and scientific development of satellite systems, and finally became participants who interacted closely with top management that was making the decisions on satellite programs. We also note that we have been involved over the decades in international cooperation that has helped to foster increasing capability and mutual complementarity of satellite systems and helped to establish coherence in the scientific utilization of satellite measurements.

2. Past evolution of satellite capabilities and their utilization

a. Technology evolution: It started by keeping it simple

The start of the satellite era is a landmark in the history of earth observation. Without a doubt, the first meteorological satellite more than 50 years ago (Suomi and Parent 1958; Menzel and Phillips 2009) was the beginning of a new exciting era that opened the eyes of mankind to many things that had been obscured before. However, it needs to be understood that the early design of the satellites was very simple and not up to the expectations that today’s users have. Several examples come to mind; for instance, data recording systems on the early Explorer satellites (Suomi and Parent 1958) were limited in storage capacity, and global coverage was limited by the number of ground sites receiving satellite data transmissions. Data processing was accomplished on paper tapes that experienced limited lifetimes. Researchers and their students processed the early data over several years; the concept of real-time processing did not exist. Satellite imagery with global coverage was realized only after laborious cutting and pasting [see Fig. 1 for an example from the early days of the Television and Infrared Observation Satellite (TIROS)].

Fig. 1.
Fig. 1.

The first global mosaic assembled from TIROS images in 1960.

Citation: Weather, Climate, and Society 7, 4; 10.1175/WCAS-D-15-0017.1

The first geostationary satellite was launched in 1966 featuring only a daytime viewing capability; atmospheric motions were soon derived (Fujita 1968). Infrared sensing useful at night was added later on. However, quantitative processing of the data remained a challenge. For instance, the request for instrument characterization (e.g., spectral response and its accuracy) of the early Meteosat imagers was met with the response from the vendor that the instrument had never been conceived for quantitative applications but should rather be considered as a camera. The first generation of Meteosat satellites did not have an adequate onboard calibration system; they were sometimes referred to as UFOs, meaning “uncalibrated flying objects.” Nonetheless, they were extremely useful also for quantitative applications thanks to vicarious calibration methods using a real-time radiative transfer model (Schmetz 1986) or aircraft measurements (Kriebel 1981). Now it appears that the information can even be usefully employed in a reprocessing effort to establish Climate Data Records (CDR) from the first generation of geostationary satellites (e.g., Lattanzio et al. 2013).

In short, quantitative usage of early satellite data required many workarounds to bypass known shortcomings. One can summarize the pioneering work of the early satellite era by saying that it was driven by a desire to demonstrate the feasibility of technology paired with the conviction that global observations of weather and climate would become a key asset as opportunities arose (Morgan 1992). It was clear that the accuracy of the measurements was limited, but that did not stop the programs from going forward. The alternative could have been to go back to the laboratory and continue technological development until measurements were adequate without actually knowing what “adequate” implied in all its facets. This would have incurred enormous costs. It is important to realize that our pioneers did a splendid job by not seeking perfection but rather by seeking excellence that actually worked. We believe that this message has deep implications for today’s satellite programs; evolution of those programs requires managers and decision-makers who are able to distinguish realistic excellence, which can be obtained within allocated budgets and time, from perceived necessity sometimes advocated by scientists.

b. Evolution of utilization: Developing applications and a user community over time

When addressing the evolution of the utilization of satellite data, it is important to note that there was not an established user community in the beginning that could help in specifying user needs. In the early times of satellite development both the vision and the technical specification were provided by a few scientists and engineers. An example is the very beginning of the meteorological satellite era largely influenced and shaped by Professor Verner Suomi; he married a viable scientific objective (measuring the outgoing longwave radiation at the top of the atmosphere) with technology (a bolometer measuring Earth-reflected and -emitted radiation) that was challenging yet could be realized within the given constraints. The resulting experiment, from Explorer 7 (Suomi and Parent 1958) started the long-term measurement of the earth energy balance that continues today (Menzel and Phillips 2009). Another example from the European side is the inclusion of a water vapor (WV) channel on the Meteosat first generation after the development of the satellite had started, an activity led by Professor Pierre Morel. This was a visionary idea to see the earth with a veil of water vapor (see Fig. 2), and the resulting applications surprised many (Morel et al. 1978) and became a well-established part of operational WV channel applications (e.g., Szejwach 1982).

Fig. 2.
Fig. 2.

An early water vapor image from Meteosat-1.

Citation: Weather, Climate, and Society 7, 4; 10.1175/WCAS-D-15-0017.1

Early geostationary satellite images, while not calibrated very well to radiances, offered useful information and immediate impact. An example comes from the NOAA National Hurricane Center (NHC), whose mission is to evacuate shorelines in advance of hurricane landfall. It was often difficult to convince beachgoers in sunny skies to escape from possible storm surge, heavy winds, and excessive rainfall; however, the image loops of the raging tropical cyclone over the Gulf or Atlantic waters helped to accomplish the task. According to the NHC director of the time (N. Frank 1980, personal communication), Geostationary Operational Environmental Satellite (GOES) imagery was the most important tool at his disposal to encourage people to leave their vacation or their homes behind and seek safety away from the shoreline. Figure 3 gives examples of GOES and Meteosat images of hurricanes from 1970 and 2003.

Fig. 3.
Fig. 3.

(top) Applications Technology Satellite 3 (ATS-3) image of Hurricane Celia from 2 Aug 1970 before landfall in the Gulf of Mexico. These images in time sequence animation provided a powerful message to the public. (bottom) This picture based on Meteosat-8 images shows a composite of the positions and evolution of Hurricane Isabel over the Atlantic Ocean from 3 to 15 Sep 2003.

Citation: Weather, Climate, and Society 7, 4; 10.1175/WCAS-D-15-0017.1

Once the satellite measurements became more established and satellite missions became routine, thus providing a sense of data continuity, a user community emerged (Johnson 1994). An early example is weather forecasters and nowcasters. Geostationary satellites providing continuous measurements became a key ingredient for monitoring tropical storms and midlatitude low-pressure systems (Purdom and Menzel 1996); they came to be operated by different satellite agencies and are now heavily used for nowcasting around the world. Over time many other operational applications have been established or are emerging: for instance, air traffic controllers use the data to monitor the displacement of volcanic ash plumes. The message here is that often opportunities offered by the new satellite observations are the start of a requirement. Generally speaking, we think that operational users are best able to formulate clear requirements for services and products they already know, while the truly novel aspects are better addressed by leading application scientists.

A user community also emerged gradually in the field of NWP. The radiances measured from the TIROS Operational Vertical Sounder (TOVS) series found on NOAA satellites provide temperature and humidity information for soundings (vertical profiles). For many years attempts had been made to use these temperature and humidity soundings in NWP analogous to information from radiosondes. NWP reliance on radiosondes propelled satellite data to be formatted into radiosonde-like characterization, but satellite radiance measurements represented contributions from broad overlapping layers in the atmosphere. Positive impact (especially in the radiosonde-rich Northern Hemisphere) of the TOVS sounding data eluded NWP centers for 15 years (Smith 1991). Finally, after a paradigm shift of using the TOVS measured radiances directly in a variational assimilation system, instead of the derived soundings TOVS, clearly positive impact was accomplished (Eyre 1997).

Tracking atmospheric motions in the time sequences of geostationary images of cloud features was also an area where positive NWP impact was sought. Early results were promising, but cloud height attribution remained a major challenge and improved NWP forecasts remained elusive. During the First Global Atmospheric Research Program (GARP) Global Experiment in 1979, a special project attempted to improve atmospheric motion vectors inferred from the geostationary imagers with cloud heights inferred from the polar-orbiting sounders using CO2 slicing. Later, after a considerable effort to properly calibrate the water vapor spectral band measurements on Meteosat, the height assignment was improved and positive impact in NWP forecasts was realized. The early steps forward in the derivation of atmospheric motion vectors (AMVs) from Meteosat are summarized in Schmetz et al. (1993); among other things, the vicarious calibration of the WV channel using a radiative transfer model in real time with radiosonde data was an essential step to improve the height assignment of thin cloud tracers. The general message from that early work is that success was here a result of incremental improvements including feature selection, tracking, and height assignment; there was no single “big leap.” On a further note, we would like to highlight the encouragement received from the European Centre for Medium-Range Weather Forecasts (ECMWF), notably from Dr. Tony Hollingsworth, to pursue the continuous improvement of AMVs. He said explicitly “give me winds, without your winds from tracking marine boundary layer clouds our assimilation system will be blind over the tropical and subtropical oceans” (T. Hollingsworth 1985, personal communication). His comment was backed by a study (Reed et al. 1987) that demonstrated the potential information on atmospheric flow from tracking clouds. We remark that this type of direct encouragement from a prime user of satellite data, such as ECMWF, was and is very important; it is highly motivating for satellite scientists and it sets the tone that, a priori, more data are good to have. It is very likely that the welcoming attitude of ECMWF to new data, notably from satellites, has contributed to maintaining the ECMWF leadership in NWP over three decades.

Often, weather forecast improvements were not realized immediately after new measurements were made available, as the previous examples demonstrate. In hindsight this tells us that the full potential of data remained untapped for a while. A convincing proof of this can be found in the reanalyses at NWP centers for climate applications (Hollingsworth et al. 2005; Dee et al. 2014). These are performed with the most recent state-of-the-art forecast systems. A comparison of the skill of the reanalysis with the skill of the real-time forecasts provides a measure of how much the use of the data in the model system improves the forecast skill. Figure 4a shows the evolution in time of the anomaly correlation of the 500-hPa height forecasts (this is a standard parameter for the ECMWF to measure impact and improvement). The increase in forecast skill is shown for different forecast ranges. Improvements are clearly discernable, and they are the result of improvements in measurements providing better and/or new data, data assimilation and modeling, and computing resources.

Fig. 4.
Fig. 4.

Annual running mean of anomaly correlations (in %) of the 500-hPa height forecasts for 3-, 5-, 7-, and 10-day forecasts for the extratropical Northern and Southern Hemispheres. (a) ECWMF operations from January 1980 to May 2013 and (b) the reanalyses from ERA-Interim from January 1979 to April 2013 and ERA-40 from January 1973 to December 2001. The shading shows differences in scores between the two hemispheres (from Dee et al. 2014).

Citation: Weather, Climate, and Society 7, 4; 10.1175/WCAS-D-15-0017.1

Figure 4b shows the forecast skills using a state-of-the-art forecast system in a reanalysis mode. Here the improvements are due only to better data becoming more available over the years. This figure from Dee et al. (2014) tells us that past forecasts could have been much better, if it had been known how to use the data as we now use them. In fact, the lion’s share of the improvements is due to advances in modeling and data assimilation (Dee et al. 2014). Of course, the advent of new and better observations is a prerequisite for subsequent improvements on the part of users. As an aside, it is interesting that the gap between the forecast skill for the Northern and Southern Hemispheres nearly vanishes from the late 1990s onward thanks to satellite data (Hollingsworth et al. 2005).

Figure 5 illustrates the point made in the previous paragraph. Experience has shown that new meteorological satellite systems are not usually exploited to their full potential when operation starts. While this is understandable, especially for the first satellite of a series, it is nonetheless a lost return on the investment. Therefore, it is recommended that the science and applications be developed beyond the foreseen products and services early in the life of a satellite system. Generally speaking, additional funding in research and development during the early stages of an operational satellite program greatly enhances the overall return on the investment.

Fig. 5.
Fig. 5.

Diagram showing the utilization of satellite data as a function of satellite lifetime hoped for in an ideal world and according to actual experience. The message is that timely research and development work toward the full exploitation of the potential of the data will increase the return on the investment by producing more mature products and services during the lifetime of a satellite system.

Citation: Weather, Climate, and Society 7, 4; 10.1175/WCAS-D-15-0017.1

There are various ways of enabling the early development of advanced utilization of products. A framework of doing so can be found in the Satellite Application Facilities (SAF) within EUMETSAT and the cooperative institutes within NOAA and NASA. The suggestion here is to be conscious of the need to rapidly develop the use of the satellite data to its full potential, especially during the early stages of a satellite program.

3. Development of recent satellite programs

Users of meteorological satellite data have developed increasingly sophisticated needs. This has inherently led to escalating complexity in the design of satellite sensors and in the associated data processing systems. In addition, there has been an order-of-magnitude increase in the data volume produced, which by itself presents significant challenges to the development of a satellite program. In light of these challenges, we reflect on processes that try to balance and meet the needs of all stakeholders.

a. Establishing user requirements: Finding the common ground between research capabilities and user requirements

At EUMETSAT, based on the experience gained from user workshops on requirements for the development of Meteosat Second Generation, a full-fledged process called User Consultation (A. Ratier 2002, personal communication) was designed, fully implemented, and widely accepted for the development of future satellite systems. Under the leadership of Alain Ratier, in his role as the director of program development at EUMETSAT, the process was used to develop Meteosat Third Generation and the follow-on generation to the current polar-orbiting MetOp satellites. The process engages the user community in a formal way from the very beginning. Considering that a series of new meteorological satellites will deliver observations for two decades or more, the approach is mainly driven by current and future needs of nowcasting, NWP, and climate applications on the basis of technology-free observations of key parameters, that is, without preempting any technological realization. It invokes a long-term vision established by research in these application areas and by expectations of the benefits of improved forecasts. Assessment of priorities also comes into the picture, from the outset, recognizing that fulfilling all requirements is not affordable. Users are heard regarding their priorities, and vice versa they get to hear about the constraints; thereby, they become part of the process and take ownership. Steps in the EUMETSAT User Consultation include the following:

  1. Application expert groups meet to address user requirements and priorities (including sampling and accuracy requirements) for observations in broad application areas dominated here by meteorological and climate applications.

  2. They write position papers on the application areas.

  3. A team of remote sensing experts then assesses to what extent demonstrated observing techniques, innovative instrument concepts, and new technologies can fulfill the established requirements; they meet with the application expert groups at regular intervals to evaluate the requirements. To leave scope for iteration, requirements are given in three categories (threshold—below this the measurement is not useful; objective—the desired performance; and breakthrough—expected to pave the way to new science and applications).

  4. A sequence of open user consultation workshops is held where the requirements, priorities, and the likelihood of fulfilling them from space are presented and discussed; this enables the broader science and operational user community to arrive at a consensus on which mission and instrument concepts should be further studied during phase 0/A engineering studies (phase 0 is the mission analysis and requirements identification; phase A is a study of feasibility). Some applications and remote sensing experts participate in the follow-up studies, forming efficient mission teams per instrument. This allows for iteration on requirements and offers compromise solutions in case of feasibility or affordability issues. An important aspect is that a dedicated team of scientists and engineers from EUMETSAT guides the user consultations.

Similar user consultation processes were started at NOAA/NESDIS, where they evolved into hosting a series of annual user conferences. The goal for the GOES user conferences is to 1) inform users of future capabilities and potential applications of the next series of NOAA geostationary satellites, the GOES-R series; 2) determine the associated user needs; 3) assess user and societal benefits of future systems; 4) develop and improve communication between NESDIS and the user community; and 5) develop and refine a process for determining and updating requirements. Over the course of conferences, several adjustments to the GOES-R instrument capabilities have been suggested and plans to realize them have been started for subsequent instrument versions.

At the World Meteorological Organization (WMO), efforts have been underway to evaluate the capabilities of the Global Observing System (GOS) against the user requirements in various applications areas (there are 12 currently; see http://www.wmo.int/pages/prog/www/OSY/GOS-RRR.html#SOG). An international Expert Team on Observational Data Requirements and Redesign of the GOS (and later the Expert Team on Evolution of the GOS) formulated a process called the Rolling Requirements Review (WMO 2001) that evaluates the capabilities of existing observing systems against user requirements, identifies gaps, and explores possibilities for improvements. The resulting implementation plan for achieving the evolution of the GOS can be found in WMO (2013).

Each form of user consultation has provided strong evidence that user engagement with leading application experts and engineers is essential in exploring technological possibilities and trading user requirements effectively against affordable technological realizations.

b. Moving from user requirements to system requirements: Working the trade space to design affordable achievable systems

Creating instrument specifications associated with the user requirements is primarily a job for the satellite science community and not the engineering community. Satellite scientists should be involved in the technical performance specifications (spectral bands, spectral resolution, field of view size, bit depth, sampling intervals in space and time, etc.) and any evaluation of the trade space when compromise must be achieved.

Part of the success of a new program will involve important short term sacrifices in requirements so that the program is affordable and still introduces new capabilities. While designing to cost is only an approximate description of this process, cost must be an important consideration.

When establishing new system capabilities, it is difficult to estimate cost. Extrapolations from past systems were often based on payload size, weight, number of spectral bands being measured, and other such parameters; these cost estimates were prone to inaccurate results. However, independent cost estimates often become a self-fulfilling prophecy as the resources estimated become the resources budgeted and hence the resources expended. Cost estimates for future ground processing systems are of limited value since they are made in the context of current technological capabilities without benefit of unforeseen future technological breakthroughs. This suggests that a practical way to design data processing systems is to fulfil the high-priority requirements first and to leave flexibility for incorporating technological advances downstream; expansion/enhancement is then possible in an affordable way.

Assessing the potential impact of future systems: Working with NWP

The desire to evolve the GOS guided by tools to assess potential impact of existing and proposed new components has been present for some time. There have been studies of how to coordinate development and utilization of a comprehensive software tool for carrying out impact experiments of future systems with Observing System Simulation Experiments (OSSEs) as well as how to prepare, maintain, and evolve a realistic database of current system capabilities. Such an undertaking requires considerable human and computer resources. Furthermore, the WMO has correctly concluded that for realistic results, it is necessary to simulate not only the particular observing system of interest, but also all other components of the GOS expected to be in place at the time of interest. For these future systems, OSSEs would also need to anticipate advances in data assimilation and model development, which is not feasible in principle. Because of these unavoidable limitations, an OSSE is only a useful complementary tool; it does not provide the unambiguous figure of merit often desired by decision-makers. We suggest that judgment on future impact is best left to experienced leading satellite applications and remote sensing scientists, who also make use of existing tools; this will contain inherent risks in decisions.

Assessing the impact of current systems is complex yet well addressed by current activities such as the regular NWP impact workshop conducted by the WMO (e.g., WMO 2012). Those impact studies provide continuous guidance to satellite operators on the current status of the relevance of the measurements to NWP. The recent NWP impact workshop (WMO 2012) has shown that the different components of the meteorological satellite observing system work in a very complementary way. An emerging question is about the adequate measure of merit when assessing the impact of a component of the observing system. It is very conceivable that an observing system that captures extreme events very well may be a good choice although its average impact in terms of the standard figures of merit (e.g., anomaly correlation of the 500-hPa geopotential) is small. Finally, we recall that there is a need to maintain a robust global observing system that necessitates a degree of redundancy.

c. Developing and implementing the product algorithms: Collaborations between the science community and industry

Postlaunch instrument performance evaluations, ground processing software implementations, and product algorithm improvements conducted over many years have revealed the following guidelines. Remote sensing scientists and applications scientists are the best source of prototype algorithms. Scientists, notably at the operational centers, must continue to drive the evolution to the operational applications software. There is a possibility for different levels of involvement of scientists during the industrial development of an (operational) applications ground segment; however, one point we would like to make emphatically is that it is not sufficient to just get specifications from scientists, pass them on to industry, and let the development happen in a disconnected way. Experience has told us that the applications software delivered by industry needs mending by scientists. During the development, a continuous involvement of scientists and engineers from the operational centers in the development of the new applications software is a prerequisite to ensure that operational centers are familiar with the software (i.e., line-by-line understanding of the code) from day 1 of operations.

d. Realizing the program through partnership: Combining the strengths of the private and public sectors

The evolution in space-based remote sensing over the last 50 years could not have occurred without the cooperation between the government, industry, and research (including academia) communities. Each of these three pillars has its own strengths. Government space agencies should have responsibility for overall organization, continuous operation, and service in support of the common good (weather, climate, and security information). Also, government must maintain critical technical infrastructure and scientific understanding needed to evolve and improve applications and services. Finally, and very importantly, government must have the technical and scientific expertise to assess, judge, and guide industrial development work. Research institutions should be involved in the development of new application concepts based on existing and new capabilities, including proof of feasibility through demonstration. Research scientists should also participate in the system design trade space discussions. Industry should realize the proven science concepts meeting the requirements on the basis of specifications and prototype applications software provided by application scientists. We think that the above-mentioned User Consultation Process at EUMETSAT (A. Ratier 2002, personal communication) goes a long way in that direction.

The economy of cost demands that operational programs buy recurrent satellites and instruments. At the same time instrument technical evolution should be encouraged, even in the absence of new user requirements. Research space agencies should lead in the effort to develop and demonstrate new technologies that measure new facets of the Earth–atmosphere system. The approaches currently in place by research space agencies, which are competitive and PI driven, do serve the purpose well.

1) Preceding operational implementation with a research demonstration

In 1980, NASA added a geostationary temperature and moisture sounding capability to the NOAA Visible Infrared Spin-Scan Radiometer (VISSR), resulting in the VISSR Atmospheric Sounder (VAS). Research and operational agencies collaborated to demonstrate the utility of time-continuous soundings as part of the Operational Satellite Improvement Program (OSIP). The VAS demonstration consisted of 79.5 days of demonstration of geosounding wherein a NOAA principal investigator led the science experiments conducted by university and government scientists and the program was supported by NASA funding. The demonstration was extended when NOAA made VAS soundings routine. OSIP made possible the introduction of geostationary soundings and paved the way for operational implementation on subsequent GOES satellites. This is an example of fulfilling the need of operational agencies to evolve their systems after demonstration of possible replacement options by research agencies.

2) Trying a new acquisition approach

During the development and acquisition of the National Polar-Orbiting Operational Environmental Satellite System (NPOESS), the NOAA and Defense Military Satellite Program (DMSP) polar-orbiting satellites were combined into one system. Expanded capabilities and considerable savings were anticipated (Winokur 1997).

As part of the new acquisition approach, all design efforts were referenced to an interagency operational requirements document (IORD) that defined capabilities in terms of environmental data record requirements, but it did not specify sensor data record requirements nor even suggest instrument approaches or specific bands. The private sector was expected to respond with instruments and algorithms. Government and research community expertise was put on the sidelines. The traditional roles of the private and public sectors were altered; unfortunately, this contributed to schedule delays and unmet user expectations. A glaring example is that the Visible Infrared Imaging Radiometer Suite (VIIRS) instrument design did not include a WV sensitive measurement, a fundamental Earth remote sensing capability. NWP forecast improvements had been realized using polar winds (Key et al. 2003) developed from Moderate Resolution Imaging Spectroradiometer (MODIS) WV measurements. But in the absence of scientific input from government and research communities, the VIIRS was rendered incapable of continuing those measurements.

In summary, the key partners in the development of new satellite systems should contribute with their strengths. They are 1) government and space agencies providing expertise to define attainable and affordable programs that also provide continuity; 2) research laboratories, academia, and advanced operational users (e.g., NWP) participating in the setting of mission objectives with an eye on achieving unmet user requirements and seizing opportunities from the new observations; and 3) industry realizing the foreseen satellite observational capabilities.

e. Fostering international cooperation and coordination: Establishing and maintaining space agency dialogue

Communities in many countries involved with the design, engineering, fabrication, implementation, and utilization of meteorological remote sensing satellite systems have become experts in their field. This has fostered international, institutional, and personal collaborations that have moved space-based remote sensing science closer to meeting the challenges in advanced meteorological satellite missions. The Coordination Group for Meteorological Satellites (CGMS) is a good forum for developing these concepts further, as they already arrange backup plans and coordinate applications algorithm development and evaluation. The need for enhancing international cooperation is large, and the main building blocks for doing so exist. The goal is for the satellite operators participating in the GOS to partition observing slots in the geostationary orbiting and overpass times of the polar-orbiting satellites; this enables global coverage with good refresh frequency. There has been considerable progress in this direction.

1) Geostationary backups

As an early example of mutual benefit through cooperation and coordination, we recall that, through agreement reached at the CGMS, it has been possible for NOAA and EUMETSAT to back each other up in case of a shortfall in geostationary satellite holdings. On two occasions the agencies were able to help each other out and thus prevent undue haste in their replacement strategies. It started in 1985, when NOAA was able to provide support to the European satellite program by making their GOES-4 satellite available to successfully support the European Data Collection System through 1988. In 1990 EUMETSAT and the European Space Agency (ESA) made it possible to operate Meteosat-3 over the Atlantic Ocean in support of NOAA weather-observing needs (de Waard et al. 1992). The positioning of Meteosat-3 at 50°W (later at 75°W), called the Atlantic Data Coverage mission, was particularly useful during the hurricane season that included Hurricane Andrew in September 1990 and precluded a rush for GOES-8 development and launch. This example set the scene for successful cooperation formalized through agreements between satellite agencies.

It has long been recognized that international cooperation is essential for achieving a more complete global observing system and for sustaining it through national mishaps in satellite programs. Formal international agreements have created a sustained cooperative network.

2) IJPS: International cooperation for global data coverage

In 1988, EUMETSAT and NOAA signed an agreement wherein the morning polar-orbiting satellite system would be provided by EUMETSAT (called MetOp; Klaes et al. 2007) and the afternoon orbit would be provided by NOAA (with NOAA-18 and -19). This was in preparation for the Joint Polar Satellite System (JPSS) that has since been realized. The cooperation helped to evolve the satellite systems and to further scientific collaboration on data utilization. Now MetOp has become a key ingredient of the space-based GOS, and it makes an essential contribution to the success of global NWP. A facet of the Initial Joint Polar System (IJPS) that deserves highlighting is that NOAA and EUMETSAT shared instruments. MetOp flies the U.S. Advanced Very High Resolution Radiometer (AVHRR), High Resolution Infrared Radiation Sounder (HIRS, on MetOp-A and MetOp-B), and Advanced Microwave Sounding Unit (AMSU-A) instruments. NOAA-18 and -19 fly the European Microwave Humidity Sounder (MHS). We see this sharing as an early example toward a potential future of instrument sharing involving even more satellite agencies.

4. Synthesis and an outlook

The previous examples and comments can be summarized as follows.

  1. New meteorological satellite programs need to be developed from the beginning in close consultation with the user community, who can offer input toward end-user requirements. However, it is important to rely on leading scientists, with practical experience and a clear vision, to guide the process. Day-to-day users provide valuable insight for keeping the established useful services, but may be limited in visualizing future advances. Early evolution of programs should be accompanied by experts in their field and be led and guided by a small team of scientists and engineers. It is strongly recommended to allocate resources early for advancing the utilization beyond the products and services committed for day 1 of operations. The goal is to reach a full use of the new satellite observations as quickly as possible (see Fig. 5).

  2. The key partners in the development of satellite systems must focus on their strengths. First, government and space agencies must have critical expertise to set out attainable and affordable objectives to assess and gauge the progress of industrial development, and ultimately to accept the deliverable. Second, research laboratories, academia, and advanced operational users (e.g., NWP) must be involved in shaping the mission objectives, in the validation process preceding the beginning of operations, and in advancing the use of the new data. Third, industry needs to have the critical know-how to realize the foreseen satellite missions and, if needed, the capability to formulate warnings that the deliverables cannot be delivered within time and within budget. It has been past experience that interaction between instrument engineers and instrument/application scientists, with a good mutual understanding, maximizes the exploitation of the design trade space.

  3. The operational utilization of new instruments (capabilities) should be preceded by a commissioning period in order to test and demonstrate the functioning of the whole system as planned; overlap with old instruments (capabilities) is strongly encouraged. Close cooperation between private-sector developers and public-sector users should be maintained, meaning that there shall be good visibility of the applications software development to the agency that eventually operates the applications software. This software code (re)engineered by industry should follow a prototype code developed by the research scientists from space agencies. Reengineering should be confined to improving data flow and software optimization. Industry is the best choice to build a complete satellite system, including the integration of the applications ground segment.

  4. The tasks for evolving the space-based GOS are many and the resources are limited, but through international coordination we can move further and achieve more. We encourage the international partners, working through the WMO, to start coherent joint planning toward realization of a more capable space-based global observing system. The coordination of individual components of the system is already in place (e.g., via CGMS), and even the mutual exchange of instruments on satellites has become reality (e.g., between NOAA and EUMETSAT). We believe the next logical step is indeed the joint planning and distribution of tasks for the global GOS from the outset. While presenting many challenges, it will provide an enormous benefit by creating elements of a complementary nature. It also makes a space-based GOS with higher potential more affordable because many recurring instruments can be built by one vendor, thus creating economies of scale. It has the further advantage that users, notably NWP centers, will find more commonality in data characteristics and hence they will have less preparation work (e.g., on forward operators, validation, and learning how to use the data). Last but not least, it creates an enhanced basis for international scientific cooperation and faster advancement toward the best use of the data.

5. Summary

From our experience, we have found that it is essential for key partners in the development of new satellite systems to contribute with their strengths as follows. Government and space agencies provide expertise to define attainable and affordable programs that enhance capability but also provide continuity. Research laboratories, academia, and advanced operational users (e.g., NWP) participate in setting mission objectives, and industry works to achieve the technological advances that realize the foreseen satellite observational capabilities.

We note that new instruments most often are not fully utilized when operations start, in part because it is difficult to formulate requirements and allocate resources that cover the full potential of those instruments. Therefore, it is recommended that operational space agencies put high priority on rapidly advancing the utilization of new measurements when they are first available.

As a thought for the future, we propose consideration of mechanisms for partitioning the components of a future global space-based observing system into elements that can be addressed by individual space agencies. A step in that direction was the exchange of instruments between NOAA and EUMETSAT for the NOAA-18 and -19 and MetOp satellites. Further movement might involve drafting a framework to address a transition from research missions to sustained operational satellite programs, a topic not addressed in this paper. While achieving this goal may appear to be very complicated at first glance, we think that the core ideas are quite simple and the realization will be a careful step-by-step process utilizing existing technology and scientific know-how on a global scale in a coordinated manner. This also offers the hope of effectively utilizing the resources of contributing countries to complementary ends so that the evolution the space-based global observing system is a global enterprise intended for the maximum benefit of all humankind. Clearly, it has the potential to maximize, within given budgets, advances in capabilities meeting the global meteorological user requirements.

Acknowledgments

We thank three reviewers for their constructive suggestions and supportive comments. Dr. Dick Dee, ECMWF, kindly provided Fig. 4. Finally, we would like to take the opportunity to thank all our colleagues, who we worked with throughout our careers, for being good colleagues.

REFERENCES

  • Dee, D. P., Balmaseda M. , Balsamo G. , Engelen R. , Simmons A. J. , and Thépaut J.-N. , 2014: Toward a consistent reanalysis of the climate system. Bull. Amer. Meteor. Soc., 95, 1235–1248, doi:10.1175/BAMS-D-13-00043.1.

  • de Waard, J., Schmetz J. , and Menzel W. P. , 1992: Atlantic data coverage by METEOSAT-3. Bull. Amer. Meteor. Soc., 73, 977983, doi:10.1175/1520-0477(1992)073<0977:ADCB>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Eyre, J. R., 1997: Variational assimilation of remotely-sensed observations of the atmosphere. J. Meteor. Soc. Japan, 75 (1B), 331338.

    • Search Google Scholar
    • Export Citation
  • Fujita, T. T., 1968: Present status of cloud velocity computations from the ATS I and ATS III satellites. Proc. Space Research IX: 11th COSPAR Plenary Meeting, Tokyo, Japan, COSPAR, 557570.

  • Hollingsworth, A., and Coauthors, 2005: The transformation of earth-system observations into information of socio-economic value in GEOSS. Quart. J. Roy. Meteor. Soc., 131, 34933512, doi:10.1256/qj.05.181.

    • Search Google Scholar
    • Export Citation
  • Johnson, D. S., 1994: Evolution of the US meteorological satellite program: 1994 Verner E. Suomi Lecture. Bull. Amer. Meteor. Soc., 75, 17051708.

    • Search Google Scholar
    • Export Citation
  • Kelly, G., 1997: Influence of observations on the operational ECMWF system. ECMWF Tech. Rep. 76, 2–7.

  • Key, J., Santek D. , Velden C. S. , Bormann N. , Thépaut J.-N. , Riishojgaard L. P. , Zhu Y. , and Menzel W. P. , 2003: Cloud-drift and water vapor winds in the polar regions from MODIS. IEEE Trans. Geosci. Remote Sens., 41, 482–492, doi:10.1109/TGRS.2002.808238.

    • Search Google Scholar
    • Export Citation
  • Klaes, K. D., and Coauthors, 2007: An introduction to the EUMETSAT Polar System. Bull. Amer. Meteor. Soc., 88, 10851096, doi:10.1175/BAMS-88-7-1085.

    • Search Google Scholar
    • Export Citation
  • Kriebel, K. T., 1981: Calibration of the METEOSAT-VIS-channel by airborne measurements. Appl. Opt., 20, 1112, doi:10.1364/AO.20.000011.

    • Search Google Scholar
    • Export Citation
  • Lattanzio, A., and Coauthors, 2013: Land surface albedo from geostationary satellites: A multiagency collaboration within SCOPE-CM. Bull. Amer. Meteor. Soc., 94, 205214, doi:10.1175/BAMS-D-11-00230.1.

    • Search Google Scholar
    • Export Citation
  • Menzel, W. P., and Phillips J. M. , 2009: Satellite meteorology: How it all started, 50 years ago. Bull. Amer. Meteor. Soc., 90, 14351436, doi:10.1175/2009BAMS2963.1.

    • Search Google Scholar
    • Export Citation
  • Morel, P., Desbois M. , and Szejwach G. , 1978: New insight into the troposphere with the water-vapor channel of Meteosat. Bull. Amer. Meteor. Soc., 59, 711714.

    • Search Google Scholar
    • Export Citation
  • Morgan, J., 1992: EUMETSAT: Objectives, role and activities. Int. J. Remote Sens., 13, 1065–1070, doi:10.1080/01431169208904179.

  • Pailleux, J., 1997: Impact of various observing systems on numerical weather prediction. Proceedings of CGC/WMO Workshop, WMO/TD 868, World Weather Watch Tech. Rep. 18, 198 pp.

  • Purdom, J. F. W., and Menzel W. P. , 1996: Evolution of satellite observations in the United States and their use in meteorology. Historical Essays on Meteorology 1919–1995, J. R. Fleming, Ed., Amer. Meteor. Soc., 99–155.

  • Reed, R. J., Klinker E. , and Hollingsworth A. , 1987: An evaluation of the performance of the ECMWF operational forecasting system in analysing and forecasting tropical easterly wave disturbances: Part II: Spectral investigation. ECMWF Tech. Rep. 60, 65 pp.

  • Schmetz, J., 1986: An atmospheric correction scheme for operational application to METEOSAT infrared measurements. ESA J., 10, 145159.

    • Search Google Scholar
    • Export Citation
  • Schmetz, J., Holmlund K. , Hoffman J. , Strauss B. , Mason B. , Gärtner V. , Koch A. , and Van de Berg L. , 1993: Operational cloud motion winds from Meteosat infrared images. J. Appl. Meteor., 32, 12061224, doi:10.1175/1520-0450(1993)032<1206:OCMWFM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Smith, W. L., 1991: Atmospheric soundings from satellites—False expectation or the key to improved weather prediction? Quart. J. Roy. Meteor. Soc., 117, 267297, doi:10.1002/qj.49711749802.

    • Search Google Scholar
    • Export Citation
  • Suomi, V. E., and Parent R. J. , 1958: Satellite instrumentation for measurement of the thermal radiation budget of the Earth. 1958 National Telemetering Conf., Baltimore, MD, Institute of the Aeronautical Sciences, 186190.

  • Szejwach, G., 1982: Determination of semi-transparent cirrus cloud temperature from infrared radiances: Application to Meteosat. J. Appl. Meteor., 21, 384393, doi:10.1175/1520-0450(1982)021<0384:DOSTCC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Winokur, R. S., 1997: NOAA satellite programs—New advances for the 21st century. WMO Bull., 46 (3), 223228.

  • WMO, 2001: Statement of guidance regarding how well satellite and in situ sensor capabilities meet WMO user requirements in several application areas. WMO Sat-26, WMO/TD-1052, 251 pp.

  • WMO, 2012: Final report of the Fifth WMO Workshop on the Impact of Various Observing Systems on Numerical Weather Prediction. WIGOS Tech. Rep. 2012-1, 25 pp. [Available online at https://www.wmo.int/pages/prog/www/OSY/Meetings/NWP5_Sedona2012/Final_Report.pdf.]

  • WMO, 2013: The implementation plan for evolution of global observing systems. WIGOS Tech. Rep. 2013-4, 110 pp. [Available online at https://www.wmo.int/pages/prog/www/OSY/Publications/EGOS-IP-2025/EGOS-IP-2025-en.pdf.]

Save
  • Dee, D. P., Balmaseda M. , Balsamo G. , Engelen R. , Simmons A. J. , and Thépaut J.-N. , 2014: Toward a consistent reanalysis of the climate system. Bull. Amer. Meteor. Soc., 95, 1235–1248, doi:10.1175/BAMS-D-13-00043.1.

  • de Waard, J., Schmetz J. , and Menzel W. P. , 1992: Atlantic data coverage by METEOSAT-3. Bull. Amer. Meteor. Soc., 73, 977983, doi:10.1175/1520-0477(1992)073<0977:ADCB>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Eyre, J. R., 1997: Variational assimilation of remotely-sensed observations of the atmosphere. J. Meteor. Soc. Japan, 75 (1B), 331338.

    • Search Google Scholar
    • Export Citation
  • Fujita, T. T., 1968: Present status of cloud velocity computations from the ATS I and ATS III satellites. Proc. Space Research IX: 11th COSPAR Plenary Meeting, Tokyo, Japan, COSPAR, 557570.

  • Hollingsworth, A., and Coauthors, 2005: The transformation of earth-system observations into information of socio-economic value in GEOSS. Quart. J. Roy. Meteor. Soc., 131, 34933512, doi:10.1256/qj.05.181.

    • Search Google Scholar
    • Export Citation
  • Johnson, D. S., 1994: Evolution of the US meteorological satellite program: 1994 Verner E. Suomi Lecture. Bull. Amer. Meteor. Soc., 75, 17051708.

    • Search Google Scholar
    • Export Citation
  • Kelly, G., 1997: Influence of observations on the operational ECMWF system. ECMWF Tech. Rep. 76, 2–7.

  • Key, J., Santek D. , Velden C. S. , Bormann N. , Thépaut J.-N. , Riishojgaard L. P. , Zhu Y. , and Menzel W. P. , 2003: Cloud-drift and water vapor winds in the polar regions from MODIS. IEEE Trans. Geosci. Remote Sens., 41, 482–492, doi:10.1109/TGRS.2002.808238.

    • Search Google Scholar
    • Export Citation
  • Klaes, K. D., and Coauthors, 2007: An introduction to the EUMETSAT Polar System. Bull. Amer. Meteor. Soc., 88, 10851096, doi:10.1175/BAMS-88-7-1085.

    • Search Google Scholar
    • Export Citation
  • Kriebel, K. T., 1981: Calibration of the METEOSAT-VIS-channel by airborne measurements. Appl. Opt., 20, 1112, doi:10.1364/AO.20.000011.

    • Search Google Scholar
    • Export Citation
  • Lattanzio, A., and Coauthors, 2013: Land surface albedo from geostationary satellites: A multiagency collaboration within SCOPE-CM. Bull. Amer. Meteor. Soc., 94, 205214, doi:10.1175/BAMS-D-11-00230.1.

    • Search Google Scholar
    • Export Citation
  • Menzel, W. P., and Phillips J. M. , 2009: Satellite meteorology: How it all started, 50 years ago. Bull. Amer. Meteor. Soc., 90, 14351436, doi:10.1175/2009BAMS2963.1.

    • Search Google Scholar
    • Export Citation
  • Morel, P., Desbois M. , and Szejwach G. , 1978: New insight into the troposphere with the water-vapor channel of Meteosat. Bull. Amer. Meteor. Soc., 59, 711714.

    • Search Google Scholar
    • Export Citation
  • Morgan, J., 1992: EUMETSAT: Objectives, role and activities. Int. J. Remote Sens., 13, 1065–1070, doi:10.1080/01431169208904179.

  • Pailleux, J., 1997: Impact of various observing systems on numerical weather prediction. Proceedings of CGC/WMO Workshop, WMO/TD 868, World Weather Watch Tech. Rep. 18, 198 pp.

  • Purdom, J. F. W., and Menzel W. P. , 1996: Evolution of satellite observations in the United States and their use in meteorology. Historical Essays on Meteorology 1919–1995, J. R. Fleming, Ed., Amer. Meteor. Soc., 99–155.

  • Reed, R. J., Klinker E. , and Hollingsworth A. , 1987: An evaluation of the performance of the ECMWF operational forecasting system in analysing and forecasting tropical easterly wave disturbances: Part II: Spectral investigation. ECMWF Tech. Rep. 60, 65 pp.

  • Schmetz, J., 1986: An atmospheric correction scheme for operational application to METEOSAT infrared measurements. ESA J., 10, 145159.

    • Search Google Scholar
    • Export Citation
  • Schmetz, J., Holmlund K. , Hoffman J. , Strauss B. , Mason B. , Gärtner V. , Koch A. , and Van de Berg L. , 1993: Operational cloud motion winds from Meteosat infrared images. J. Appl. Meteor., 32, 12061224, doi:10.1175/1520-0450(1993)032<1206:OCMWFM>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Smith, W. L., 1991: Atmospheric soundings from satellites—False expectation or the key to improved weather prediction? Quart. J. Roy. Meteor. Soc., 117, 267297, doi:10.1002/qj.49711749802.

    • Search Google Scholar
    • Export Citation
  • Suomi, V. E., and Parent R. J. , 1958: Satellite instrumentation for measurement of the thermal radiation budget of the Earth. 1958 National Telemetering Conf., Baltimore, MD, Institute of the Aeronautical Sciences, 186190.

  • Szejwach, G., 1982: Determination of semi-transparent cirrus cloud temperature from infrared radiances: Application to Meteosat. J. Appl. Meteor., 21, 384393, doi:10.1175/1520-0450(1982)021<0384:DOSTCC>2.0.CO;2.

    • Search Google Scholar
    • Export Citation
  • Winokur, R. S., 1997: NOAA satellite programs—New advances for the 21st century. WMO Bull., 46 (3), 223228.

  • WMO, 2001: Statement of guidance regarding how well satellite and in situ sensor capabilities meet WMO user requirements in several application areas. WMO Sat-26, WMO/TD-1052, 251 pp.

  • WMO, 2012: Final report of the Fifth WMO Workshop on the Impact of Various Observing Systems on Numerical Weather Prediction. WIGOS Tech. Rep. 2012-1, 25 pp. [Available online at https://www.wmo.int/pages/prog/www/OSY/Meetings/NWP5_Sedona2012/Final_Report.pdf.]

  • WMO, 2013: The implementation plan for evolution of global observing systems. WIGOS Tech. Rep. 2013-4, 110 pp. [Available online at https://www.wmo.int/pages/prog/www/OSY/Publications/EGOS-IP-2025/EGOS-IP-2025-en.pdf.]

  • Fig. 1.

    The first global mosaic assembled from TIROS images in 1960.

  • Fig. 2.

    An early water vapor image from Meteosat-1.

  • Fig. 3.

    (top) Applications Technology Satellite 3 (ATS-3) image of Hurricane Celia from 2 Aug 1970 before landfall in the Gulf of Mexico. These images in time sequence animation provided a powerful message to the public. (bottom) This picture based on Meteosat-8 images shows a composite of the positions and evolution of Hurricane Isabel over the Atlantic Ocean from 3 to 15 Sep 2003.

  • Fig. 4.

    Annual running mean of anomaly correlations (in %) of the 500-hPa height forecasts for 3-, 5-, 7-, and 10-day forecasts for the extratropical Northern and Southern Hemispheres. (a) ECWMF operations from January 1980 to May 2013 and (b) the reanalyses from ERA-Interim from January 1979 to April 2013 and ERA-40 from January 1973 to December 2001. The shading shows differences in scores between the two hemispheres (from Dee et al. 2014).

  • Fig. 5.

    Diagram showing the utilization of satellite data as a function of satellite lifetime hoped for in an ideal world and according to actual experience. The message is that timely research and development work toward the full exploitation of the potential of the data will increase the return on the investment by producing more mature products and services during the lifetime of a satellite system.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 1226 469 33
PDF Downloads 832 253 21