The paucity of long-term observations, particularly in regions with heterogeneous climate and land cover, can hinder incorporating climate data at appropriate spatial scales for decision-making and scientific research. Numerous gridded climate, weather, and remote sensing products have been developed to address the needs of both land managers and scientists, in turn enhancing scientific knowledge and strengthening early-warning systems. However, these data remain largely inaccessible for a broader segment of users given the computational demands of big data. Climate Engine (http://ClimateEngine.org) is a web-based application that overcomes many computational barriers that users face by employing Google’s parallel cloud-computing platform, Google Earth Engine, to process, visualize, download, and share climate and remote sensing datasets in real time. The software application development and design of Climate Engine is briefly outlined to illustrate the potential for high-performance processing of big data using cloud computing. Second, several examples are presented to highlight a range of climate research and applications related to drought, fire, ecology, and agriculture that can be rapidly generated using Climate Engine. The ability to access climate and remote sensing data archives with on-demand parallel cloud computing has created vast opportunities for advanced natural resource monitoring and process understanding.
Climate Engine enables users to process, visualize, download, and share climate and remote sensing datasets with a simple web connection, thereby overcoming common big data barriers.
Climate and weather affect all sectors of society at regional to local scales. However, the paucity of long-term observations in many parts of the globe provides a constraint on the utilization of data for applied use and scientific research. To address the need for place-based data, a number of operational gridded climate and meteorological datasets have been created (Daly et al. 1994; Mitchell et al. 2004; Abatzoglou 2013; Thornton et al. 2014; Oyler et al. 2015), in addition to remote sensing datasets that are freely available and are being increasingly used. However, the accessibility of these data to researchers, decision-makers, and the general public are limited because of challenges related to computational requirements, data storage, and software needed to work with large volumes of data.
A good example of these limitations can be illustrated with current drought monitoring. Climate data are the primary basis for operationally providing information about the degree and intensity of drought conditions through the use of drought indices (Hobbins et al. 2016; Svoboda et al. 2002; McKee et al. 1993). Climate-based drought indices are complemented by relatively fine-resolution satellite remote sensing data (Anderson et al. 2007; Brown et al. 2008; Wang and Qu 2007). These data can be calculated at their native spatial and temporal resolutions—often at spatial scales from 30 m to 12 km and temporal scales from hourly to weekly. However, operational products that are summarized for decision-makers are typically available at much coarser spatial (e.g., climate division) and temporal (e.g., monthly) resolution. Moreover, many operational web-based products are static and offer limited options for interacting with the data, customizing analyses to specific periods for summaries, or acquiring the digital data.
Recognizing these limitations, recent web applications have focused on providing on-demand and dynamic visualization, extraction, and processing of precomputed data (Berrick et al. 2009; Eberle et al. 2013; Teng et al. 2016). New computing technologies, where massively parallel processors are collocated with data collections, allow for on-demand and on-the-fly generation of custom data products and visualization, thereby avoiding many limitations of the past (Moore and Hansen 2011; Baumann et al. 2016; Yang et al. 2017). The development of a cloud-computing web application for on-demand processing and visualizing climate and remote sensing data is motivated by current web application limitations, and by climate and natural resource scientist and manager needs related to drought, ecology, and agriculture that can be addressed through advanced processing and visualization of Earth observation archives.
This article outlines the development of a free web application called Climate Engine (http://ClimateEngine.org) (Fig. 1) that uses Google’s parallel cloud-computing platform, Google Earth Engine (Gorelick et al. 2017), to enable users to process, visualize, download, and share various global and regional climate and remote sensing datasets and products (e.g., anomaly maps and time series) in real time. Climate Engine helps overcome many of the data storage and processing limitations that are common to researchers, practitioners, and stakeholders. The development of Climate Engine is detailed through a brief discussion of the software application development and design. Several case study applications of Climate Engine are highlighted to illustrate its ability to generate maps and time series for rapid analysis and visualization, and to support advanced natural resource monitoring and process understanding.
The Climate Engine web application development and design philosophy was centered around providing the ability for users to perform on-demand mapping and time series visualization and analyses that are customizable, and where map and time series results can be downloaded in common file formats, or shared via web URL links (see sidebar about “Application and development design”). Datasets and variables available within Climate Engine are derived from existing Google Earth Engine image collections that are consistently updated with minimal latency (approximately 1–16 days) operational data (Table 1). Additional datasets, variables, and calculations are continually being added by user request and as Climate Engine evolves.
The Climate Engine web application is hosted on the Google App Engine web server, while the source code is hosted on a GitHub repository for version control and source code management. The source code is divided into two parts: the front end, which the user sees on the web page; and the back end, which is where the requested data are formed and processed.
The front-end display of Climate Engine is viewed in a web browser and is constructed using the Twitter Bootstrap 3 Cascading Style Sheets (CSS) web framework for the navigation bars and tabs, and the overall responsive design of the site. The display contains a form for users to customize their requests and a section for displaying the response (a map layer or a time series figure with data).
The map-layer display illustrates user-requested raster output on a Google map. Climate Engine provides the ability for the user to customize the map layer (e.g., scale and color palette options) and to place optional vector images (e.g., KML, polygons, Google Fusion Tables) atop the map layer to aid in geographical orientation or to be used for spatial averaging.
The beauty of this application framework is that the requests can be made from anywhere a web browser has an Internet connection, all major computing is performed using the thousands of processors via Earth Engine, and results are returned to the device for display and/or download.
Climate Engine offers both mapping and time series analysis options. Users are able to choose specific product types (remote sensing or climate), datasets (different satellite platforms or gridded climate datasets), variables (from precipitation to vegetation indices), and common calculations (climatologies or anomalies) and statistics (mean, median, maximum, minimum, total) for customized time periods. In the mapping view, users are able to modify the map layer displayed on the Google map by adjusting the color palette, transparency, and value ranges; to perform masking; and to add vector layers to the map. Users can also request values from the map or download rectangular regions of the map layer in Georeferenced Tagged Image File Format (GeoTiff) (Fig. 1). In the time series view, users are able to choose from one of three types of time series visualizations for data covering either a point location or an area average: daily values (or native temporal resolution of dataset), interannual summaries of values over a defined period, or values within a year compared to statistics from other years. Data from multiple point locations or from multiple variables can be compared at the same time. Users can dynamically interact with the resulting scalable vector graphics (SVG) figure to view values at data points, zoom in on the time series figure, toggle the display of series data, download the figure in common image formats, and download the data in comma-separated values (.csv) or Excel (.xls) file format. Climate Engine provides easy access to remote sensing and climate archives by pairing cloud-computing capabilities and a web application, thereby avoiding the computational expenses of storage and processing such large datasets.
CASE STUDY APPLICATIONS.
We demonstrate the potential of Climate Engine to both the research community and decision-makers by highlighting several recent case studies related to climate, drought, fire, ecology, and agriculture. Map and time series figures shown in the case studies were all computed and downloaded using Climate Engine and edited (i.e., projection and color modifications) to create publication-quality graphics; however, readers can visit http://ClimateEngine.org/bams to replicate these case study maps and time series in real time.
Recent winters have featured a pattern of anomalously warm air over the Arctic and anomalously cool temperatures over portions of midlatitude continents, potentially a by-product of Arctic sea ice loss and internal atmospheric variability that is associated with a polar vortex (Overland and Wang 2016; Waugh et al. 2016). For example, the Arctic-wide temperature anomaly in January–February 2016 was 5.0°C above normal (Overland and Wang 2016), whereas the eastern half of North America was exceptionally cool. Climate observations are rather sparse in the Arctic. However, finescale (1 km) temperature anomalies can be detected through remotely sensed land surface temperature (LST), from the Moderate Resolution Imaging Spectroradiometer (MODIS) sensor on board the National Aeronautics and Space Administration (NASA)’s Terra and Aqua satellites. Climate Engine’s mapping tools were used to compute and visualize anomalous surface temperature and snow cover maps in the following examples. Figure 2a illustrates the median LST anomaly for January–February 2016 relative to a 2000–15 baseline climatology, showing a large swath of Alaska, Canada, Greenland, and Siberia with LST from 6° to 10°C above normal. Figure 2b highlights a similar example, but for January–March (JFM) 2015, where the impact of the polar vortex on LST is clear and compelling.
Last, Greenland experienced unusually high LST in April 2016, with some locations in the interior approaching 20°C above the 2000–15 average (Fig. 2c), prompting an early commencement for ice melt in the southwestern portion of Greenland in mid-April that resulted in widespread ice loss (Mottram 2016). Figure 2d shows the unusually low ice cover [via the normalized difference snow index (NDSI) from MODIS] in early April along the southwestern portion of the Greenland ice sheet and coincides with early melting (Mottram 2016).
Drought is a sustained imbalance of supply (precipitation) and demand (i.e., evaporative demand). The demand side of drought is often overlooked but can be equally important as supply. In addition to common supply and demand drought indices available on Climate Engine (Table 1), the evaporative demand drought index (EDDI; Hobbins et al. 2016; McEvoy et al. 2016) is also available and was computed with meteorological forcings from gridded surface meteorological (gridMET) data (Abatzoglou 2013) as inputs into the American Society of Civil Engineers’ Penman–Monteith standardized reference evapotranspiration (ET0) equation (Allen et al. 2005). EDDI is an effective drought metric due to two distinct feedbacks between regional evapotranspiration (ET) and ET0: a complementary relationship under water-limited conditions (extended drought) where ET and ET0 vary in opposite directions (Bouchet 1963; Morton 1969), and a parallel relationship under energy-limited conditions and at the onset drought (Budyko 1974; Hobbins et al. 2016).
The summer of 2012 Great Plains drought stands out as one of the most extreme drought events in instrumental records (Hoerling et al. 2014), with estimated total economic losses (largely from the agriculture sector) of $35 billion (U.S. dollars). The Climate Engine–derived June–August (JJA) EDDI map shown in Fig. 3a highlights the large spatial extent of this drought, with extreme and exceptional drought categories stretching from the Canadian to the Mexican borders and encompassing approximately two-thirds of the continental United States. Using the interannual time series options and the spatial averaging feature of Climate Engine, the time series illustrated in Fig. 3b shows the accumulated summer ET0 anomaly averaged over Missouri in 2012 was greater than 200 mm above average and approximately 100 mm greater than the previous record set in 1980.
The MODIS normalized difference vegetation index (NDVI) and LST anomalies are especially useful for evaluating regional vegetation stress due to drought. During the warm season, LST is largely a function of the ET rate due to evaporative cooling (i.e., latent heat flux). If ET is relatively low, then the LST will be relatively high, and vice versa. Figures 3c and 3d illustrate reduced NDVI and increased LST during the summer of 2012 Great Plains drought due to reduced vegetation vigor and ET, respectively. Having multiple indicators of drought that are readily accessible through on-demand cloud computing and web visualization is extremely useful for better understanding the drivers and impacts of drought from multiple perspectives and disciplines (i.e., land surface energy balance, vegetation, near-surface boundary layer).
Snow drought is a term that has been recently used to describe the lack of snow depth or coverage that occurs simultaneously with near-normal or above-normal precipitation conditions. A snow drought occurred during the winter and spring of 2014/15 over the northwestern United States, featuring record-low snowpack conditions in the Cascades and northern Rocky Mountains even though respective precipitation amounts in many areas were at or above normal (Cooper et al. 2016; McEvoy et al. 2016). To illustrate this example, maps of the standardized precipitation index (SPI; McKee et al. 1993), EDDI, and the MODIS-derived NDSI anomaly for December 2014–March 2015 were generated with Climate Engine (Figs. 4a–c).
Fire danger indices are used operationally to assess wildfire potential for short-term wildland fire business decision-making (e.g., large fire potential) and are used by the research community as a numerical measure of fuel aridity. Four sets of fire danger indices using the U.S. National Fire Danger Rating System (Cohen and Deeming 1985) are computed daily from the gridMET data and are available via Climate Engine, including the energy release component (ERC). ERC is a proxy for daily potential fire radiative energy that integrates temperature, precipitation, humidity, and solar radiation over the preceding several weeks and exhibits strong links to the occurrence of very large fires (Riley et al. 2013) and seasonal burned area (Abatzoglou and Williams 2016), particularly across forested regions. A map of ERC anomalies and a time series of ERC averaged over Boise County, Idaho, for JJA 2016 are shown in Figs. 5a and 5b, respectively. The Pioneer fire started on 18 July 2016 in the Boise National Forest and burned a total of 76,000 ha, making it one of the largest fires of the 2016 western U.S. fire season. The fire primarily burned during a period of well-above-normal ERC values for much of the latter half of July and August, including making large fire runs during the first couple days of August and the last couple days of August when ERC values were well above normal, approaching the 90th–95th percentiles for the calendar day. To visualize the areal extent of the burn, Fig. 5c illustrates the 30-m-pixel-resolution Landsat-8 NDVI anomaly from 18 July to 22 September 2016 relative to the Landsat-5, -7, and -8 climatology (1984–2015), clearly highlighting the burned area as regions of decreased vegetation greenness.
The use of gridded climate and remote sensing products within the ecology community is becoming commonplace as relevant datasets have become available and accessible, and as requirements for long-term monitoring are becoming standard for permitting land and water development projects. Ecological modeling and monitoring typically require finescale information at long time scales (i.e., ∼30+ years). The combination of gridded high-resolution climate data and the Landsat satellite image archive has catapulted ecological-focused research, such as change detection at fine and large spatial scales (Cohen and Goward 2004; Wulder et al. 2012; Roy et al. 2014).
Long-term monitoring of groundwater-dependent ecosystems (GDEs) for baseline assessments and water and land use impacts analyses is a central focus area for many western U.S. federal, state, and nongovernmental agencies. These agencies must adhere to regulations and requirements, including environmental assessments and monitoring related to sage-grouse habitat, groundwater development, and mining. A compelling example of how Climate Engine can be used for advanced GDE monitoring is shown in Fig. 6, which illustrates the coincident increase and decrease in Landsat-derived summer vegetation vigor (i.e., NDVI) beginning in 2002 for respective agricultural and adjacent spring areas located in eastern Nevada and western Utah (Figs. 6b and 6c). These changes are a consequence of groundwater pumping for agriculture, lowering of the groundwater table, and drying of the spring (Halford 2015; Huntington et al. 2016). Paired with gridMET water-year precipitation and ET0 computed with Climate Engine, Fig. 6d shows that lowering of the groundwater table has markedly changed the vegetation response to precipitation (PPT) within the spring area. Also evident is the complementary relationship between NDVI and ET0 with PPT, a novel illustration showing how atmospheric demand, supply, and vegetation response are inherently linked.
Monitoring agricultural vegetation vigor is important for assessing water use, irrigation performance, crop yields, and drought impacts and for reviewing and litigating water rights, transfers, and disputes. All of these issues are receiving high-priority attention in the western United States and in other water-limited environments around the world. The use of high-resolution satellite imagery is needed to accurately characterize spatial and temporal variations in crop productivity, phenology, and water use over large areas. Given free access to the >30-year Landsat archive combined with Google Earth Engine, rapid field-scale assessments can be readily produced using Climate Engine.
An example of a field-level analysis, where Climate Engine was used to generate maps of growing season maximum NDVI from Landsat for 2011 and 2015 over the Tulare Lake basin in the Central Valley of California is shown in Fig. 7. Figures 7a and 7b clearly illustrate the large amount of fallowing that occurred in 2015 due to the multiyear drought. Melton et al. (2015) estimated that over 2,000 km2 of Central Valley cropland was fallowed in 2015, approximately twice that of 2011. A unique and very powerful feature of Climate Engine is the ability to extract field-level time series information related to vegetation vigor for anywhere around the globe using predefined polygons, user-uploaded Keyhole Markup Language (KML) files, or by dynamically drawing polygons on the map to define areas of interest. The latter option is applied in Fig. 7c to examine field-level crop phenology stages (dormant, green-up, full cover, and senescence/harvest periods) from 2011 to 2015, clearly showing the fallowed land in 2015.
Africa’s Sahel region is the semiarid zone just south of the Sahara, but north of the humid tropical zone of Africa. The agriculture, livestock, and human villages and pastoralists in this region are heavily dependent on rainfall. During late 2015 to early 2016, a strong El Niño contributed to regional shifts in precipitation in the Sahel region. Significant drought across Ethiopia resulted in widespread crop failure and more than 10 million people in Ethiopia required food aid (U.S. Department of State 2016). Global precipitation datasets can be used to detect local to regional anomalies in precipitation as a tool for devising early-warning systems for drought-related impacts, such as famine. Whereas most available global precipitation products have coarse spatial resolutions, the Climate Hazards Group Infrared Precipitation with Station (CHIRPS) dataset (Funk et al. 2015) provides quasi-global (50°S–50°N, 180°E–180°W) pentad (5 day) precipitation totals at ∼4.8-km resolution from 1980 to the present, and is ideal for monitoring shifts in regional precipitation and drought in areas with limited observations, such as Africa.
Climate Engine–derived precipitation anomalies from CHIRPS for late 2015–early 2016 depict large portions of Ethiopia, particularly pastoral regions, received less than half of average rainfall during their primary growing season (Fig. 8a). The multiscalar and multiproduct nature of Climate Engine is demonstrated by evaluating NDVI anomalies at regional and field scales with MODIS and Landsat datasets, respectively (Figs. 8b and 8c), and is especially useful for supporting famine early-warning efforts and assessments of on-the-ground impacts.
DISCUSSION AND CONCLUSIONS.
Cloud computing of environmental datasets is rapidly changing the way researchers and practitioners are conducting research, making applications, and planning for long-term application sustainability (Zhang et al. 2010; Hansen et al. 2013; Pekel et al. 2016; Yang et al. 2017). The motivation behind Climate Engine is to enable users to quickly process and visualize large datasets of climate and satellite Earth observations for advanced monitoring and process understanding and to improve early warning of drought, wildfire, and crop-failure risk at spatial scales relevant to scientists and decision-makers alike. Application features include on-demand mapping of environmental monitoring datasets, customizable analyses, time series and statistical summaries, downloadable digital data, and URL link sharing that reproduce results in real time.
What makes Climate Engine unique is the unprecedented access for processing, visualizing, and analyzing Earth observation datasets with a simple web connection, overcoming computational burdens of big data, and providing the ability to download or share results instead of downloading and processing entire data archives. A valid argument is that the science community should be focused on data discovery, providing answers, and creating new useful tools, rather than devoting time to downloading and processing entire archives on local or research institution network computers. Cloud-computing platforms, such as Google Earth Engine, allow us to move from archives to answers very efficiently, bypassing all the downloading and processing paralysis of yesteryear.
A primary challenge is integrating new cloud-based research findings, products, and applications into decision-making activities. The modular nature of Climate Engine allows for easy integration of additional climate and remotely sensed datasets that are added to Google Earth Engine collections. Future directions of Climate Engine involve working with collaborators to identify and integrate additional spatial averaging domains (e.g., watershed boundaries, grazing allotments, ecological units) for generating summary products; developing climate and remotely sensed datasets that best address community needs; and performing extensive and detailed outreach, stakeholder engagement, and trainings. Such stakeholder engagement and coproduction of knowledge will ultimately enhance Climate Engine as a sustainable resource for advanced natural resource monitoring and process understanding, and lead to taking the next step—translate data into decisions.
Funding for this study was provided by a Google Earth Engine faculty research award (unrestricted gift from Google), the U.S. Geological Survey 2012–17 Landsat Science Team (G12PC00068), the U.S. Bureau of Land Management Nevada State Office (L13AC00169), the U.S. Geological Survey–Great Basin Cooperative Ecosystem Study Unit (G15AC00137), and the National Integrated Drought Information System of the National Oceanic and Atmospheric Administration (AB-133E-16-cQ-0022). The authors thank FEWS NET and the Climate Hazard Group at the University of California, Santa Barbara, for their assistance in making CHIRPS precipitation products available through Google Earth Engine and for providing input on famine early warning applications.