The past decade has seen increased adoption of cloud platforms to scale performant storage, compute, and network across big data challenges. It has become increasingly apparent that these advances hold great potential for increased understanding of our planet. Key to this understanding is the availability of open environmental data in a manner that is both eminently accessible and performant.
Making environmental data open is the key focus of the NOAA Big Data Program (BDP), now known as NOAA Open Data Dissemination (NODD). Since 2015, NODD has been building offerings of publicly available cloud datasets through its trusted data broker, the Cooperative Institute for Satellite Earth System Studies (CISESS). To do so, the program leverages relationships with Amazon Web Services (AWS), Microsoft Azure, and Google Cloud Platform (GCP) to host NOAA environmental data on the cloud. These data are freely accessible to anyone via the internet (Fig. 1). As of the AMS town hall panel in January 2022, NODD hosts more than 15 PB of NOAA data, including popular datasets such as GOES-R, JPSS, NEXRAD level II, World Ocean Database, and GFS/GEFS. In many cases, these data are more performantly available through NODD and, in some cases, the only source of near-real-time, publicly available products.
Leveraging cloud platforms to support open environmental data access at scale has given NODD and its data broker CISESS unique insights into how these data are used through unique engagements between public and private partners in industry, government, and academia. These relationships have driven development of NODD and have engendered broader discussions around future priorities in facilitating access to open environmental data. A key focus that has arisen in discussion over the past year is on interoperability. With more and more environmental data being made publicly available, linking these data in accessible, performant formats is becoming a challenge.
This work was supported by National Oceanic and Atmospheric Administration through the Cooperative Institute for Satellite Earth System Studies under Cooperative Agreement NA19NES4320002.