The Coordination Group for Meteorological Satellites (CGMS) provides an international forum for the exchange of technical information on geostationary and polar-orbiting meteorological satellite systems. In 2013, the CMGS established the Socioeconomic Benefits Tiger Team (SETT) to develop a credible methodology and common terminology for articulating the socioeconomic benefits of satellite observing systems, and to explore the most effective ways to communicate the benefits to decision makers and stakeholders. As part of its first years’ activities, the SETT gathered examples of socioeconomic studies across all member organizations. This article describes key elements of these studies, and identified eight key themes that are presented. We welcome additional collaborations to identify opportunities to incorporate socioeconomic best practices, integrate these into additional or subsequent phases of work on new instruments and satellites, and develop recommendations for the way forward for the broader meteorological community.
The Coordination Group for Meteorological Satellites (CGMS) provides an international forum for the exchange of technical information on geostationary and polar orbiting meteorological satellite systems. CGMS’s goals are to coordinate and support operational weather monitoring, forecasting, and climate monitoring in response to requirements specified by national and international agencies including the World Meteorological Organization (WMO), a specialized agency of the United Nations (UN). The CGMS focuses on coordinating satellite systems using an end-to-end perspective, including protection of in-orbit assets and supporting users to facilitate and develop shared access to and use of satellite data products in applications. Increasingly, CGMS members must justify investments in new satellite systems requiring operational satellite agencies to understand and explain how these systems will benefit their nations’ economies and societies.
In 2013, the CGMS established the Socioeconomic Benefits Tiger Team (SETT) to develop a credible methodology and common terminology for articulating the socioeconomic benefits of satellite observing systems, and to explore the most effective ways to communicate the benefits to decision makers and stakeholders. To accomplish these goals, the SETT recruited expertise from across the CGMS membership, including from its academic partners, and compiled members’ experiences and ideas on how to evaluate the usefulness of meteorological data. The Tiger team has participants from NOAA and NASA in the United States, the European Organization for the Exploitation of Meteorological Satellites (EUMETSAT) and the WMO, as well as the Japanese and the Chinese Meteorological Agencies. This article describes the activities of the SETT and provides information about how readers can participate in its activities.
During the 43rd CGMS Plenary held 20–22 May 2015 in Boulder, Colorado, the SETT chair reported on the team’s first two years of activities and plans for the future. The SETT held two workshops; compiled relevant studies conducted by member countries and noted the commonalities across all gathered socioeconomic studies, including eight key themes (see Fig. 1), which are summarized here; and identified needed expertise. The SETT held a third workshop on 7 October 2015, at the Organization for Economic Co-operation and Development (OECD) in Paris, France, where SETT members finalized the design and implementation strategy for a case study on “Understanding and Assessing the Value of Improved Satellite Data for the Users of Operational Sea Ice Products and Information,” and submitted a guidance document for use by CGMS members considering their own socioeconomic benefit studies.
Although there is extensive literature on valuing information derived from scientific and technological systems, including by economists Jeffrey K. Lazo and Molly K. Macauley, the sponsors of studies focused on determining the value of satellite data are typically national and international meteorological satellite organizations themselves. Because a great deal of the work required to conduct an analysis on the value of information is done in the preliminary framing of the study, including the definition of its goals, objectives, and the audience, the SETT has focused on collaboration across meteorological agencies on learning how these critical context-setting steps can be done well.
CONTEXT IS ESSENTIAL.
The most useful socioeconomic benefit studies focus on a specific question or objective. Prior to conducting a study, the sponsor must identify needed resources, time constraints, and the intended audience for the study. The sponsor must also understand what questions face the audience and how they will use the study’s results to make decisions.
Some of the specific topics and questions addressed by the studies presented to the SETT include “possible contribution of meteorological satellites to energy savings in Japan” and “what is the potential use of Earth observations for volcanic ash advisories?” These demonstrate the need to go beyond the impact of the observation on numerical weather prediction and focus on specific applications.
METHODS ARE CRITICAL.
To answer the study’s specific question(s), the sponsor must understand and explain the logical connections between the observations and socioeconomic benefits. Determining the value in use of an observation or system requires a demonstration of how and why the data are a component in decision-making. End user engagement must inform the methodology selected. For example, when planning for new sea ice data products for improved navigational safety in the Arctic, it is necessary to have an understanding of the data latency, spatial resolution, and data format requirements of navigation systems of commercial vessels. If this information is not considered during the development of the system, the data may not be used by the community.
UNDERSTANDING RELATIVE IMPACT OF SATELLITE OBSERVATIONS IS CRUCIAL.
The WMO and the operational agencies are placing great efforts in using Observing System Experiments or Forecast Sensitivity to Observations to understand the role of satellite observations in the context of all observing systems and their relative impact on numerical weather prediction. Analysts can use this information to extrapolate the socioeconomic benefits derived from a particular observing system. For example, the WMO can generate an indicative “impact per cost” ranking by dividing the impact by the estimated annual cost for an observing system.1
QUANTITATIVE AND QUALITATIVE METHODOLOGIES ARE BOTH VALID APPROACHES.
While a quantitative approach to estimating socioeconomic benefits may be more convincing, there are caveats. When employing a quantitative approach, the sponsor must ensure the analysis is sufficiently rigorous to withstand scrutiny by economists and social scientists. In addition, sponsors of quantitative studies must balance the need for economic rigor with the ability to represent the full value of a system; recognizing conservative estimates of benefits may lead to undervaluing the data or products. Qualitative assessments may allow for the description and valuing of benefits important for the community but that cannot be easily quantified.
The sponsors must be able to communicate the results along with any underlying socioeconomic assumptions or limitations so that decision makers understand the findings and their context. It is important to state which factors are measured and which are excluded, as well as to state any assumptions. Clarifying terminology across multiple disciplines is an important first step in this process.
For example, a recent study sponsored by EUMETSAT identified the quantitative factors used in the analysis, including within the three main benefit areas identified in Fig. 2.2 Equally important, the study states that while the contribution to safety of life is an invaluable benefit for society and decision makers, it was not quantified in the analysis. Understanding what the analysis measures and excludes is critical to allow decision makers to interpret the results accurately.
THE VALUE OF PERFECT INFORMATION MAY NOT BE WORTH THE COST OF ACQUISITION.
Cost-benefit analyses are increasingly time-consuming and costly as one moves down the value chain. A decision about the scope and methodology of a proposed study must factor in the cost of conducting the study. One of the reasons costs can increase as one moves down the value chain is the need to acquire and standardize diverse datasets.
DATA AVAILABILITY MATTERS.
Study planning should examine the need for ancillary data, and determine if the data needed to undertake a socioeconomic study are publicly available or commercially restricted. Building relationships with user communities can help facilitate access to ancillary datasets. For example, calculating the costs attributable to volcanic ash advisories requires data from the airline industry, and estimating the contribution of Earth observations to a malaria early warning system requires access to public health data. Developing relationships with end users is an important aspect of planning a study.
INTERDISCIPLINARY EXPERTISE IS REQUIRED.
The most robust studies bring together experts from across the physical and social sciences over the life time of the project, allowing for repeat analyses and recalculation of benefits. An interdisciplinary team that has worked together to clearly define terminology for socioeconomic benefit studies can increase the value of the overall result. For example, undertaking a socioeconomic analysis of the value of Earth observation information to a malaria early warning system in Botswana required a breadth of expertise—including social scientists and public health experts as well as operational and research agency experts. All partners in the analysis needed to agree upon how to define the impact of the satellite data on improved malaria detection that was being quantified and how it related to individuals, communities, institutions, and the overall economy. Much of that work focused on defining terminology that all could agree upon.
OPERATIONAL AGENCIES CAN LEVERAGE RESEARCH AGENCY PERSPECTIVES AND EXPERTISE.
Cooperation across operational and research agencies combined with increased collaboration with end users will shed light on different perspectives and uses of observations, providing a more complete picture of the full value chain. The SETT is working with end users and economists to complete a case study of the end-to-end value of an Earth observation/product system to help determine best practices that can inform future studies. This case study adopts a macro approach to weather beginning with the observing systems as a whole, and then drills down to demonstrate the value of information within a specific application. Figure 3 illustrates this concept.3
The CGMS Socioeconomic Tiger Team (SETT) is developing information on the strengths and weaknesses of various methodologies, translating common terminology for articulating the socioeconomic benefits of satellite observing systems, and exploring the most effective ways to communicate the findings. We welcome additional collaborations to identify opportunities to incorporate socioeconomic best practices, integrate these into additional or subsequent phases of work on new instruments and satellites, and develop recommendations for the way forward for the broader meteorological community.
FOR FURTHER READING
Presented by Stephan Bojinski, WMO, to the CGMS SETT Workshop, 24–25 April 2014, "Cost-Benefit Studies for Observing Systems."
Benefits of NASA Earth Science and Earth Observing Satellites, Lawrence Friedl, NASA Headquarters.
Presentation to the World Meteorological Organization 12th Consultative Meeting on High-level Satellite Matters.