1. Introduction and background
The 557th Weather Wing (557 WW) and its predecessors [i.e., HQ Air Force Weather Agency (AFWA), Air Force Global Weather Central (AFGWC), and USAF Environmental Technical Applications Center (ETAC); see the appendix for a list of acronyms] have been home to the DoD’s only operational regional and global land data analysis systems with lineage dating back to the transition to operations of the “Soil Moisture Project” in January 1958 (Cochrane 1981). Originally, the Soil Moisture Project (Sturm 1977) was developed to monitor crop growing conditions over much of eastern Europe and Asia for the U.S. government. Subsequently the initial products, produced for limited regions, provided estimates of soil moisture, precipitation, and snow depth, providing the weather support for real-time, qualitative assessments of agro-meteorological conditions and quantitative agricultural yield and condition computations for the U.S. Department of Agriculture, as well as other agencies involved with agro-meteorological analysis or decision making (Cochrane 1981). This soil moisture analysis system has been significantly updated three times, once in 1974, once in 1991 with the implementation of a global capability known as the Agriculture Meteorological Model (AGRMET), and finally in 2009 with the implementation of the Land Information System (LIS; Kumar et al. 2006; Peters-Lidard et al. 2007; Kumar et al. 2008a).
Typical of updates in most operational systems, testing improvements to the AGRMET algorithms often required months to years of parallel runs in order to fully evaluate software and science upgrades, even after software changes were completed. This was partially caused by the lack of a large historical archive of boundary condition data in order to test the software, as well as due to AGRMET’s non-modular software design. Any updates to core land surface model components from partner organizations, updates to input forcing datasets, or any other modifications, recommended as part of the broader scientific community would often require months to years of internal testing, combined with potentially months of coordination with end users in order to update the operational software. Further, while the AGRMET software relied on a number of remotely sensed datasets to support computing the meteorological forcings used to drive the analysis, including precipitation and snow analyses, the software lacked the modern assimilation capabilities being introduced into the land data assimilation community at the time. These challenges combined to create research-to-operations (R2O) gaps that often would have been impossible to reduce without partnering on a common software system with other state-of-the-art Land Data Assimilation System (LDAS) development. Recognizing this critical gap, the USAF Weather Agency (AFWA, now 557th Weather Wing) partnered in 2004 to develop a common LDAS, using the LIS framework as the core software capability, to replace AGRMET. This paper describes the technical details of this transition and some lessons learned in enabling the R2O transition.
This paper summarizes the history of the early USAF Agriculture Meteorology (Agromet) system based on Cochrane (1981) and AGRMET modeling capability summarized by Moore et al. (1991), background information important to understand the development of the USAF-LIS system. Additionally, this paper describes the development, integration, and implementation of the USAF-LIS system completed in 2009. Further, the relationship created between the USAF and NASA to develop the USAF-LIS capability was instrumental in the formation of a long-term partnership, which led to a number of additional investigations and capability integrations the USAF sponsored. These integrations both improved the LIS capabilities within the USAF and enabled the broader community to benefit, including those published and available in the scientific literature (see https://lis.gsfc.nasa.gov/publications for a comprehensive list). This paper will tie those developments together and describe how the LIS development was accomplished in terms of a deliberate scientific planning process.
The partnership, the integration of AGRMET components into LIS, and resulting advancements of the LIS framework within this partnership have significantly contributed to the broader scientific community by providing an advanced land data assimilation software system. The satellite assimilation capabilities, real-time and coupled land–atmosphere forecasting prediction technologies, and integration of new and/or improved land surface models into LIS were all supported by this partnership. This has supported improvements to weather forecasting systems, global drought and flood applications, supported agriculture impacts analysis and forecast products, and climate applications by groups that have adopted and use the LIS framework. Further, the partnership has delivered improvements to a broad array of end users that that receive and have integrated LIS results into their decision support applications, including global food and water security related efforts, global crop production estimates, the Famine Early Warning System Network, military users, and the development of a global flood awareness and prediction system. Further, the knowledge and understanding of many of those downstream uses helped guide the partners when determining what capabilities to integrate into the LIS system. Eylander (2013) and a previous unpublished requirements review in Eylander et al. (2007) describes the extensive analysis of end-user needs and requirements that guided the development of goals for further developing the LIS software into an advanced land data assimilation and land modeling system. This in-depth review of documented needs and requirements by the end user community provided the understanding needed to both continue advancing the capabilities as well as defend it within defend continued funding for the program.
2. History of land characterization with the USAF Agriculture Meteorology (AGRMET) System
The USAF production of soil moisture analyses began in 1958 at USAF Environmental Technical Analysis Center (Cochrane 1981) with the implementations of the “Soil Moisture” and “Agromet” programs. Initially, soil moisture products were computed using Thornwaite indices to generate a 10-day (“decade”) estimate of soil moisture, based on global precipitation observations and snow depth/cover measurements that were also aggregated and gridded, and included with the soil moisture products. The early products coverage domains were primarily for two of the world’s largest crop growing regions. In 1974, the Agromet system was upgraded to include a Penman-based computation of evapotranspiration potential (ETP) and the limited regional coverage was expanded to cover many of the Northern Hemisphere crop growing areas. The Global AGRMET system was implemented in 1990 (Moore et al. 1991), producing global soil moisture and temperature analysis using the Oregon State University (OSU) land surface model (LSM; Mahrt and Ek 1984; Mahrt and Pan 1984) as the primary physics package used to compute the soil moisture and soil temperature analyses. Interestingly, the name change from AGROMET to AGRMET was largely driven by then-FORTRAN limitations in the length of variables (variable names were limited to six letters or letter/digit combinations).
During this time, the USAF also partnered with the National Oceanic and Atmospheric Administration (NOAA) National Weather Service (NWS) National Meteorological Center Development Division (now National Centers for Environmental Prediction Environmental Modeling Center; NCEP/EMC), NOAA Office of Hydrology, and Oregon State University to support the expansion of OSU’s LSM scheme into what is now known as the Community Noah Land Surface Model (Noah LSM; Chen et al. 1996; Ek et al. 2003). The Noah LSM name initially represented an acronym (NOAH) of the partnering organizations, but later versions were renamed to the Community Noah LSM (Noah) after version 3.0 was released to represent the broader participation of the scientific community input to model upgrades. The Noah LSM was initially implemented in the AGRMET system in November 1999 with many subsequent upgrades up through Noah 2.7.1 in 2005. The USAF AGRMET analysis system, using the Noah LSM, was used to compute a number of global, gridded surface hydrology and energy balance products, including soil moisture, soil temperature, precipitation estimates, snow depth, snow water equivalent, snow cover products, and surface latent and sensible heat fluxes. The products generated by AGRMET were distributed widely within the U.S. government, primarily to support partner-agency global numerical weather prediction model initialization, climate modeling communities, and the U.S. Department of Agriculture global crop production estimation. Internally within the USAF, the products were also used to support satellite retrievals of clouds within the Cloud Depiction and Forecast System, using the AGRMET surface temperature analysis to initialize a short-term prediction of surface temperature (Kopp 1995) used within infrared cloud retrieval algorithms.
During the 2000–03 timeframe, the USAF evaluated coupling the AGRMET system to the National Center for Atmospheric Research (NCAR) Mesoscale Model Version 5 (MM5) regional prediction system (Gayno and Wegiel 2000), as a way to initialize the new coupled land component within the MM5; which included inline versions of the Noah LSM to support land–atmosphere surface energy exchange to improve the accuracy of mesoscale weather forecasts. The resulting evaluation of the AGRMET-MM5 coupling highlighted challenges using a relatively coarse-resolution, polar stereographic, global AGRMET system to initialize a regional domain, higher resolution weather prediction model with the primary issues being the resolution and grid structure the AGRMET system used in producing products. The AGRMET polar stereographic, hemispheric grids, described by the USAF in “mesh” grid terminology, were “eighth mesh” or 48-km grid resolution (Hoke et al. 1985) products, true at 60°N, and required data reprojection in order to match up with the MM5 (and subsequently, WRF) regional domains that were either on Lambert conformal or Mercator projected model grids at resolutions from 45 km spatially and finer. The USAF polar stereographic grids were unique compared to other standard polar stereographic projections, in that the grid was rotated slightly so Washington D.C. was the center longitude. The combination of factors and required modification of products during the reprojection and regridding of the datasets added additional error (beyond model error). An updated AGRMET system supporting output for both higher resolutions and different projections, operating for both global and regional grids, was envisioned that could support both requirements.
The AGRMET system was executed operationally four times daily for the 0000, 0600, 1200, and 1800 UTC production cycles with a start time 4.5 h after cycle, and computed two, 3-hourly analyses per cycle, representing the cycle time plus the preceding 3-h analysis, resulting in eight 3-hourly analyses at 0000, 0300, 0600, 0900, 1200, 1500, 1800, and 2100 UTC. During the 1200 and 0000 UTC cycle times a 12-hourly analysis was computed and a 24-h (daily) average soil moisture, snow, and temperature analysis was also created during the 1200 UTC cycle.
The AGRMET software was engineered as FORTRAN 77 code with several distinct executables and a UNIX shell environment script driver to link the executables. The executables were responsible for separate components of the model, including generating the precipitation estimates, surface weather estimates, executing the land model, and postprocessing the AGRMET output. While modern for the time it was developed, the software lacked scalability and modularity to generate higher resolution output, the ability to execute with parallel processing, and lacked the ability to make use of higher resolution satellite observations of land surface features.
In 2004, the USAF began testing a newer version of the AGRMET model modified to operate on the 16th mesh grid, a USAF-specific, polar stereographic grid at 24-km grid-space resolution at 60° latitude. The primary objective was to support the higher resolution products requested by the AFWA numerical weather modeling team to enable initialization of the land surface model within the coupled MM5 and WRF mesoscale modeling systems. However, the lack of modularity within the AGRMET software and lack of a meteorological forcing archive capability enabling retrospective testing of model updates limited the ability to test model updates. Therefore, the only capability for offline model testing was to run a parallel, real-time, often multiyear spinups of the model to evaluate physics, parameter, and software updates in order to compare it to the existing operational version of the model. This was often challenging because the development server was not maintained 24 h a day, 7 days a week like the production servers, so any system outage could significantly interrupt model testing. The 16th mesh version of AGRMET was run for three years in parallel, on a development server, with the operational version to spin up the higher-resolution soil moisture profile products. Occurring simultaneously with the 16th-mesh AGRMET spinup and model evaluations, NASA introduced the USAF land team to a newer land data assimilation software system called NASA Land Information System (LIS), capable of supporting both global and regional domains on a resolution-configurable grid producing soil hydrology and energy balance products. At the time, the LIS system included the Noah LSM version (Noah 2.7.1) used in the operational AGRMET system, while also supporting other LSMs from the broader scientific community. Additionally, the LIS had demonstrated capability to incorporate high-resolution, satellite-based observations of land surface parameters from the NASA Moderate Resolution Imaging Spectroradiometer (MODIS) sensor, considered a pathfinder to the Visible and Infrared Imaging Radiometer Suite (VIIRS) sensor planned for the National Polar-orbiting Operational Environmental Satellite System, the precursor national joint DoD–NOAA satellite acquisition program to the Joint Polar Satellite System, now managed by NOAA/NASA. Additionally, the LIS system supported grid-space resolutions approaching 1 km spatially, potentially enabling the USAF to begin producing soil moisture products that could better support U.S. Army mobility requirements, something the coarser resolution products were unable to support. The continuing discussions between the USAF and NASA on potentially integrating AGRMET into the LIS software baseline resulted in a partnership to collaborate on the development of a merged AGRMET-LIS system.
The meteorological forcings used within the AGRMET system included data from a number of other modeling systems, including the NCEP Global Forecast System, USAF Snow Depth Analysis (SNODEP) model, USAF Cloud Depiction and Forecast System Version II World Wide Merged Cloud Analysis (WWMCA), and gridded precipitation analysis from the geostationary satellite precipitation analysis (GeoPRECIP) system. Additionally, the AGRMET system incorporated SSM/I–SSM/IS satellite observations and surface gauge observations for both the snow depth and precipitation analysis. As part of the transition from AGRMET to the LIS, some software and science components from AGRMET needed to be integrated into LIS, enabling LIS to directly replace AGRMET in the USAF operations. The initial NASA-LIS system developed under NASA Earth Science Technology Office sponsored effort (Kumar et al. 2006) resulted in a more modular LDAS software that could potentially support USAF’s need for a higher resolution, global-and-regional domain capability meeting the existing global operational needs, while also supporting the need to initialize the USAF regional Weather Research and Forecasting (WRF) coupled regional atmospheric prediction system with matching resolution, grid, and LSM physics. However, the initial versions of LIS had limited support for meteorological forcing, shortwave and longwave radiation, and snow depth data sources; a fully operational version of LIS needed to be able to ingest the raw meteorological datasets and compute those forcing components internally in order to fully replace the AGRMET system. The next section describes those components of the AGRMET system that were integrated into the LIS system.
3. The AGRMET system terrain forcings
The specific terrain characteristic options used within AGRMET, including the datasets generally accompanying the Noah LSM and supplied by either the NCAR or NCEP along with distributions of the Noah LSM source code, or combined with additional datasets, are provided by NASA as part of the LIS software distribution. Global land use and land cover information was from the U.S. Geological Survey (USGS) EROS Data Center 24-class land cover products derived from NOAA Advanced Very High Resolution Radiometer satellite data and downloaded from the NCAR website as a supporting dataset to the Noah land surface model. Soil texture data were from the UN Food and Agriculture Organization maps (Reynolds et al. 2000) combined with U.S. Department of Agriculture Natural Resources Conservation Service Digital General Soil Map of the United States (STATSGO) data (Miller and White 1998). The FAO and STATSGO soil texture data were combined and mapped onto the AGRMET eighth mesh grid system according to the Zobler (1986) soil type classification scheme. Further, terrain elevation and land–water mask information were based on USAF products as described in Hoke et al. (1985), while terrain slope categories were from Zobler (1986) and processed onto the USAF polar stereographic, eighth mesh grids. The Noah LSM also required monthly surface albedo (Matthews 1983), based on, and monthly green vegetation fraction, based on Gutman and Ignatov (1998), as parameters for vegetation and surface albedo computations, with the datasets acquired from the NCEP website. Finally, two static datasets including the maximum snow albedo (Robinson and Kukla 1985; Ek et al. 2003) and soil bottom temperature (Chen and Dudhia 2001) are required by the Noah LSM, also acquired from the NCEP.
a. Surface longwave and shortwave radiation
The AGRMET system computed longwave and shortwave energy estimates based on hourly, global fractional cloud cover analyses provided by the USAF Cloud Depiction and Forecast System Version-II WWMCA (Gustafson and d’Entremont 2000; d’Entremont and Gustafson 2003; HQ AFWA 2005; d’Entremont et al. 2016; Fig. 1). Figure 2 outlines the longwave and shortwave algorithms as flow charts. The WWMCA products include two hemispheric, once-hourly estimates of total cloud cover, and up to four layers of cloud information on a polar stereographic grid projection at a resolution of 24 km (true at 60° latitude), based on operational satellite information from global operational geostationary weather satellites plus NOAA, Defense Meteorological Satellite Program (DMSP), and European Organization for the Exploitation of Meteorological Satellites polar-orbiting satellites. The cloud cover information includes estimates of fractional cloud coverage per grid cell (percent grid cell is cloud covered), while the layered grids also include cloud top and base height estimates, along with cloud type (e.g., cumulus, stratus) and the age of the cloud satellite information used to create the cloud products for each grid cell (pixel times).
The USAF World Wide Merged Cloud Analysis (WWMCA) Total Cloud Cover Analysis for all cloud layers. The WWMCA was available on two hemispheric, polar-stereographic grids and available hourly. The WWMCA computes up to four layers of cloud cover information, including cloud type and fractional cloud amount per grid cell.
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
Flowchart diagram of the AGRMET computation of (left) longwave and (right) shortwave radiation fields, using the USAF WWMCA and SNODEP analysis systems (circa 2004). The purple ovals represented code that computed a product, and the yellow rectangles represent the resulting products in that computation. The arrows represent the flow of information through the entire process. Green rectangles represent cloud cover products that are ingested from the USAF World Wide Merged Cloud Analysis (WWMCA). The resulting longwave (at left) and shortwave (at right) radiation products are represented by the cylinders at the bottom of each flow diagram.
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
The cloud cover information is combined with the atmospheric temperature, humidity, and pressure information to compute the downward longwave radiation, based on algorithms combined from Idso (1981) and Wachtmann (1975). The shortwave estimates combined the WWMCA data with the SNODEP products to compute the atmospheric transmissivity and reflection of the shortwave energy, and combined with a backscatter estimate, based on a method originally described in Shapiro (1987).
b. Precipitation analysis
The precipitation analysis (Fig. 3) capability within AGRMET included a complex method of integrating a number of precipitation observation types, aggregating and processing global gauge and present weather observations to create 3-hourly estimates (matching the model output time step), and computing and blending multiple satellite-based estimates of precipitation together to create a global estimate. The satellite sources used in the precipitation estimate include those derived from global Geostationary-based infrared sensors and polar-orbiting DMSP SSM/I–SSM/IS microwave satellite-based rain-rate estimates.
AGRMET global precipitation analysis 3-hourly accumulated precipitation product at 1200 UTC 14 Sep 2008. The units are in total precipitation (mm) over the 3-h time period. The peak precipitation area over the United States was associated with the remains of Hurricane Ike across Missouri and Illinois, with a peak value of 41.0 mm (3 h)−1.
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
The merged precipitation analysis in AGRMET was weighted heavily toward observations from METAR and SAO surface gauge reports available from the World Meteorological Organization Global Telecommunication System. Figure 4 illustrates each of the sources of gauge, remote sensing, and other datasets used to comprise the AGRMET precipitation blending algorithm. The AGRMET software was engineered to retrieve gauge observations in ASCII-text format from an internal USAF computing database. The retrieved global gauge observations included 6-, 12-, and 24-hourly precipitation reports and perform some initial observation filtering of specific locations with recurring questionable observations (a.k.a “blacklist” observations list), as well as a few additional quality checks based on extreme precipitation amounts, etc. The checking of observations and/or observation quality requires a significant amount of manual investigation in order to understand and properly decode country and/or region-specific reporting practices, such as not adhering strictly to WMO precipitation reporting standards, or to monitor changes in reporting practices, which would impact the number of observations included in either the database and/or the precipitation analysis.
Wiring diagram describing the AGRMET merged precipitation process. Observations and estimates from satellite and other sources are listed across the top in the green parallelogram shapes. The actions of either an algorithm or process is represented in the ovals, with the resulting product represented in the in the yellow squares below the ovals. After the results were merged, the final three steps represented by the bottom three squares fed data to a Barnes optimal interpolation process to produce the final merged, 3-hourly precipitation estimate. The column letters are used in the text for reference.
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
In addition to precipitation gauge observations, the AGRMET system evaluated present weather observations as part of the gauge inclusion process to help break the precipitation reports into 3-hourly estimates. The AGRMET system used the present weather observations, or reports of precipitation type without an associated precipitation amount due to the lack of a rain gauge. The present weather reports would be used to estimate a precipitation amount, assigning accumulation amounts based on reported weather category (e.g., drizzle) reported. Sites reporting drizzle would provide a lower amount than a site reporting rain, a thunderstorm, or other type of precipitation category.
Gauge and present weather observations were applied to both the grid cell associated with the latitude and longitude of the reporting site and adjacent grid cells if those adjacent cells lack in situ observations. The precipitation amounts in those adjacent grid locations would be reduced by a linear rate according to a “spread radii” parameter. The spreading radii was an adjustable value that could be increased or decreased, and the final value of the spread observation value was also considered during the blending of gauge observations with the satellite estimates. The spread radii adjustment was not regionally modifiable, meaning one spread radii value was applied to the entire modeling domain and not adjusted regionally based on observation density or other factors. The spread radii were important to estimate precipitation in remote areas of the world where gauge observations were sparse.
In addition to the present weather estimates, AGRMET included precipitation estimates from three satellite-based sources, visually illustrated in Figs. 4c–g. The primary sources included SSM/I and SSM/IS passive microwave rain rate environmental data records (EDRs; represented in Fig. 4g); however, the impact of those datasets was limited to tropical locations out of concern for the rain rate validation in non-tropical locations. Second, the USAF computed a precipitation estimate based on geostationary satellite-based infrared satellite data using an internally developed algorithm based on an algorithm from Vicente et al. (1998), termed the GEOPRECIP analysis (Figs. 4e,f). The GEOPRECIP output (Fig. 5) were included in the AGRMET merged precipitation analysis in regions between 50° latitude north and south. Last, the AGRMET system computed a precipitation estimate using the WWMCA data which was only used in regions north or south of 50° latitude (Figs. 4c,d). Finally, if no precipitation amount from any of the other sources was available, the AGRMET relied on a gridded climatological precipitation dataset developed by the USAF Combat Climatology Center (now known as the 14th Weather Squadron). Based on testing, this precipitation dataset was rarely, if ever, used.
Example merged, hemispheric, gridded geostationary infrared satellite dataset used as input to the GEOPRECIP algorithm, along with Northern Hemisphere GEOPRECIP algorithm results on the USAF eighth mesh grid.
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
The blending of the precipitation data sources (gauges, multiple satellite sources, climatology, etc.) was accomplished using a Barnes objective analysis technique (Barnes 1964) scheme. Each of the precipitation sources were given a rank from one through five, with gauge observations having the highest rank, and climatology the lowest, with the final merged precipitation analysis representing the blending of the observations relative to the ranking/weighting and availability.
The AGRMET software included an additional precipitation algorithm that combined the CDFSII WWMCA data with a precipitation climatology to compute a precipitation estimate in areas where other remote sensing-based datasets lacked coverage, essentially in regions beyond 50° latitude. The lack of gauge observations in many regions, especially north of 50°N require additional datasets to support precipitation estimation. In this case, adapting the output from the USAF global cloud analysis system to support the diagnosis of precipitation provided at least some information. The WWMCA-based precipitation algorithm relied primarily on a combination of climatological cloud coverage variables, including an interpolated daily cloud cover fraction, median cloud cover percent, overcast cloud cover values, for each grid cell as diagnosed from the cloud climatology. The comparison of the current hourly WWMCA value to the climatology provided a precipitation factor that was multiplied against the climatology, if the current total cloud cover fraction was higher than the climatological value, then assigning a precipitation amount based on that climatological exceedance. Because the precipitation value was computed from the total cloud cover amount and did not include any information about cloud type or other cloud variables, the estimate was generally a low value.
The merged global precipitation analysis resulted from the combination of the multiple observational methods, relying initially on gauge observations when and where available combined with the remotely sensed observations, and engineered to take advantage of as much data as possible while accounting for locations or model grid cells where observational data were lacking. Figure 6 illustrates the impact of the various remote sensing methods, with the CDFSII estimate used primarily poleward of 50° latitude and mainly influencing precipitation estimates in the Northern Hemisphere, SSM/IS estimates used in tropical locations, and GEOPRECIP used elsewhere equatorward of 50°. When any one of the sources of precipitation were not available, including observations, climatological precipitation would serve as a backup source. In most cases, climatology was only used poleward of 50° when the CDFSII system lacked timely data due to extended satellite outages.
Reference grid showing the use of nongauge precipitation estimates in the USAF Agriculture Meteorology (AGRMET) system.
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
c. Surface weather parameters
AGRMET combined surface weather observations (temperature, humidity, winds), available globally as METAR format weather reports, with gridded products generated by global numerical weather prediction models in order to generate the global gridded observations needed to support the land models. A diagram visually describing the combining of datasets is included in Fig. 7. The observations were blended with the 0- and 3-h forecast fields to generate a complete grid of surface weather parameters used as forcing to the LSM. The gridded numerical weather prediction (NWP) data were from one of two sources, either the NCEP Global Forecast System (GFS) or the U.S. Navy Fleet Numerical Operations Center Navy Operational Global Atmospheric Prediction System (NOGAPS) models. The GFS was the primary model used for the AGRMET system, with the NOGAPS model providing a backup source of surface weather observations if the GFS had not been received for a specific length of time. The model was built to read in the 0- and 3-h forecasts from the GFS every model cycle (0000, 0600, 1200, and 1800 UTC model runs). The AGRMET code was built with backup source capabilities in order to maintain operations when datasets did not arrive on time, or at all for some cycles. In the case of missing GFS data, caused by a number of potential issues including internal database availability, data unavailability due from NOAA, or many other potential reasons, the AGRMET software would use a previous GFS forecast. If the interruption in GFS data flow to the AGREMT system continued for more than 24 h, the surface meteorological fields from the NOGAPS would be used until the GFS products were available again.
Wiring diagram representing the AGRMET surface meteorological processing. NWP data from either the GFS (or NOGAPS if GFS not available) were blended with observations. The NWP data from all the atmospheric isobaric levels below 500 hPa are used to capture the relevant “surface” in areas of variable terrain. Observations are blended using the Barnes optimal interpolation method and the final products include a global surface pressure, wind, temperature, and relative humidity estimates.
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
The AGRMET system blends the observations with the NWP output using a weighted Barnes (1964) optimal interpolation scheme, which also included a correction to the observations for terrain elevation. The data were linearly interpolated between the time periods in order to create the internal 15-min time steps.
d. Global snow depth estimates
The USAF computes a global snow depth analysis separately from the AGRMET system and provides the global snow cover, depth, and snow age products as a separate analysis product. The snow depth analysis model, as operated in 2008, is described in a USAF technical report (Hall 1986). Simultaneously with the development of the LIS-AGRMET system, the USAF partnered with NASA to upgrade the SNODEP model algorithms and software under a project titled the AFWA-NASA Snow Algorithm (Foster et al. 2011). The AGRMET system relied on snow depth and snow cover grids, inserted during the 1200 UTC daily cycle, to reinitialize the snow depth and snow water equivalent fields in the AGRMET. The SNODEP analysis model generated a global snow depth estimate using in situ observations and remotely sensed estimates from satellite data sources, including snow depth observations available from global METAR and SAO report locations and snow cover maps from DMSP satellite imagery. Initially, the primary snow depth information in the model was based on the observations with the satellite imagery limited primarily to snow cover; however, later improvements after the initial LIS implementation included an increased reliance on the snow depth estimates provided by satellite imagery.
4. The DoD–NASA joint LIS testbed
To support long-term LIS testing and comparisons to AGRMET, the USAF land modeling team established a testbed environment on the Department of Defense (DoD) High Performance Computing Modernization Program (HPCMP) U.S. Navy Distributed Supercomputing Resource Center (DSRC) system. The HPCMP provides DoD users and approved partners access to the DSRC computing resources operated by the services and provides computing hours for approved projects and computational support. The DoD LIS testbed was initially established within the Navy DSRC, due to the similarities of the computing architecture to the then-AFWA production environment, which were using IBM’s proprietary Unix operating system, known as AIX. The initial testbed activity was to begin archiving all the static and dynamic forcing data required to successfully execute the AGRMET system, including the surface observations, satellite data, SNODEP, WWMCA, and Global Forecast System products. The forcing archive contains all the raw input datasets described above in order to conduct model spinups and evaluations, and is considered stable with very few data gaps beginning 1 December 2005 through present. It continues to be maintained by the 557 WW and NASA for LIS development, integration, and testing. The use of the DSRC reduced the burden on the internal development computation system at AFWA, while also enabling the use of the long-term archive.
The DSRC system consists of both an archive server where data can be stored and retrieved from a permanent tape archive system, and a computing cluster. The archive server capacity supports a significant number of DoD projects and is nearly limitless in its capacity supporting development of a large meteorological forcing archive for long-term LIS testing without concern for exceeding system capacity. During the initial testing of the LIS system, the computing systems were IBM Power4+ processors consisting of 368 nodes with eight 1.7-GHz processors per node, including all the necessary computing libraries for massive parallel computing support and software batch submission scripts required to fully test the LIS system and compare the output against the production AGRMET system, and provided sufficient computing capacity to support long-term (e.g., multiyear) spinups of the land model. Since the initial testbed development, the HPCMP systems have continued to modernize with newer computing technologies, including HP, SGI, Cray, and other massively parallel computing clusters.
While the system was initially established in order to facilitate joint testing of the initial software capability by both the AFWA and NASA LIS teams, the USAF has continued to populate the testbed with data from the AGRMET and LIS systems to support long-term spinup and testing of LIS. Updates to AGRMET and/or LIS often require long-term model spinups, or continuous cycling runs in which the previous model cycle output serves as the “first guess” or initial conditions for the next model cycle. The testbed data archive, with a consistent feed of input forcings needed to execute the AGREMT system, supports the need to test any updates to the internal land surface model on the resulting LIS output. In terms of changes to Noah, this often-required testing over multiple months and occasionally years in order to fully evaluate changes. The addition of the projects at the DoD HPCMP added a significant additional capability for the LIS team, enabling us to test the LIS software on similar architecture to what the AFWA was running, but also having a nearly unlimited data archive capability to store months of forcing datasets. After developing the scripts to push all of AGRMET’s input data over to the HPCMP, we started populating the LIS testbed with archived forcing data on 1 December 2005 and have continued to populate the testbed ever since. All forcing data, including observations, gridded satellite datasets, atmospheric model data, snow depth products, and others are archived on the system. In addition to establishing the testbed, we were able to obtain user accounts for the NASA team on the system, which enabled them to test the software on an AFWA similar architecture prior to delivery. The testbed is now considered a joint testbed facility and as of this paper houses over 13 years of archived forcing data for long-term testing of the LIS software for the USAF and Army support.
The DoD LIS testbed also serves as an unofficial repository for storing production AGRMET and LIS output from the production systems in order to facilitate comparisons between the production system and development versions of the LIS source code.
5. LIS-AGRMET system testing and integration
In addition to migrating the meteorological forcings and other components described above into the LIS software baseline, integration of the LIS system into USAF Weather operations required many additional software changes in order to execute operationally as an automated component of the USAF Weather modeling enterprise. The LIS software needed to be updated to include the USAF-specific forcing datasets described previously beyond those already included within the LIS system, (e.g., NCEP Global Data Assimilation System (GDAS), NASA’s GEOS-5 data assimilation system). Specifically, the AGRMET software tools to read and compute radiation, precipitation, surface forcing, and snow forcing fields needed to be merged into the LIS software. Fortunately, both the LIS and AGRMET modeling systems were built upon similar FORTRAN core capabilities, were both version-controlled and managed through a similar software repository management system, and the AGRMET software was internally well documented and structured, thereby not requiring a significant rewrite of either software system in order to merge functionality and integrate baselines. The AGRMET software modules were inserted into the LIS baseline within a “restricted” LIS-based module due to a then-existing agreement between the USAF Weather Agency and the U.S. Department of Agriculture that restricted the releasability of AGRMET products, due to their historical use in global commodity assessment. Further, the software modifications needed to prepare the LIS software to function in an operational computing environment in an automated manner, included updating the software’s decision-tree matrix to automatically account for missing input datasets (e.g., switching to previous GFS model runs if the current cycle is not available), use of backup meteorological input forcing datasets and/or climatology data, updating error messaging in case of model failure during execution, and making modifications to account for the specific computing environment. This matured the software from an advanced research tool to something that could also execute operationally without significant manual intervention.
The completed software changes were introduced as configurable items within the LIS configuration file (the lis.config file), including a specific option that introduced a new running mode option, the “AGRMET,” now “AGRMET Ops” mode. The “AGRMET Ops” running mode directs the LIS software to read in the raw USAF base forcing datasets including GFS, surface observations, multiple precipitation data sources, snow depth information, and those described above, and to execute LIS as it would operate in a production environment. The “retrospective” mode operates on archived forcing fields already computed and archived in LIS readable files (e.g., GRIB, NetCDF, binary). Once the software updates were completed, resulting in a merged LIS-AGRMET capability, an initial benchmark test was completed comparing the output to the production AGRMET results for a period of 2 December 2005–28 February 2006, with data supplied from the DoD LIS testbed, in order to compare the integrated computation of forcing products between the AGRMET and LIS systems from each of the input sources described in the previous section. The comparisons examined the fit between the LIS-AGRMET forcings computed in the new system against operational AGRMET-computed forcings, and also evaluated the results of both models against observations from specific locations as a way of ensuring the forcing datasets were integrated correctly into the LIS system. Longer-term spinups evaluating land surface model performance were not considered as part of this initial benchmarking process due to already similar implementations of the Noah LSM in each code baseline, but were considered later during longer-term internal testing within the USAF to evaluate similarity.
Four global intercomparison sample dates were chosen for evaluating the gridded results at the beginning (2 December 2005) and end (28 February 2006) of the benchmarking period as well as on the winter solstice (21 December 2005) (Fig. 8) and a “random” intermediate date (20 January 2006). In general, the output flux intercomparisons are consistent with the input radiation differences, showing sensible (Qh) and latent (Qle) heat flux differences were less than 10 W m−2, which are considered acceptable given the differences in grid structure. The soil temperature differences peaked in mountainous regions, and those differences were isolated and determined to be caused by interpolation of initial conditions from the USAF mesh-based polar stereographic grid to the new latitude- and longitude-based grid within LIS. The LIS-AGRMET comparisons to AGRMET for surface temperature, relative humidity, surface wind speed, and accumulated precipitation were in broad agreement, with very minor differences, again related to changing the computing grid from polar stereographic to a latitude–longitude (lat–lon)-oriented grid, and potentially some of the differences caused by machine precision, since the LIS fields were being executed on a different computing system than the AGRMET production environment.
Comparisons between LIS and AGRMET computed (a) shortwave radiation (W m−2) and (b) downward longwave radiation (W m−2) forcing fields valid at 1200 UTC 21 Dec 2005. Fields are computed as AGRMET minus LIS.
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
There were larger differences in the computed shortwave and longwave radiation, again associated with the grid processing difference within the LIS code. The largest differences between the LIS computed radiation and AGRMET radiation values occur at points associated with cloud cover in the WWMCA data. The WWMCA products were available on the USAF 16th-mesh polar stereographic grid (24-km resolution valid at 60° latitude), similar to the AGRMET grid projection. The differences in computed radiation forcing required additional evaluation to consider the most effective method to integrate the radiation software into LIS. There were several approaches considered during the AGRMET integration into LIS to minimize the grid interpolation issues and to help minimize radiation computation differences, including: 1) first interpolating the cloud data to the lat–lon grid and then computing the radiation results; 2) computing the radiation results completely in polar stereographic grid space and re-projecting the results; or 3) a hybrid approach. The first approach of re-projecting the cloud data to the lat–lon grid, synchronizing the radiation computation with the surface albedo estimation using the lat–lon datasets, albedo climatology, and Noah’s snow water equivalent climatology initially appeared to be the most straightforward approach; however, this resulted in differences of up to 400–500 Wm−2 over certain grid points.
The second approach, which computed the radiation on the polar stereographic USAF grid configuration and reprojected the final radiation output, resulted in maximum differences of about 10–40 W m−2. The second option differences were tied to the lack of a global field of snow water equivalent in the lat–lon projection space. Finally, the hybrid approach was also tested and evaluated where the albedo is computed as in the first option. In this case, instead of interpolating the cloud data, the software followed the flowchart in Fig. 2 through the computation of transmissivity and reflectivity on polar stereographic cloud products, then those intermediate products were interpolated to the lat–lon grid and the remaining computations were accomplished on the lat–lon grid. This hybrid approach produced maximum differences of about 100–150 W m−2, with the pattern of differences showing that these large values are occurring at points where cloud data exists.
The radiation processing uses the cloud data (amounts, and types) to compute radiation flux, taking into account the backscatter effects by computing transmissivity and reflectivity for each layer, and using a modified albedo (based on the climatology and snow water equivalent from the land model). There are a number of model parameterizations within the algorithm primarily corresponding to the three-layer cloud data. While the implementation of the AGRMET radiation in LIS could not be exactly duplicated, because of the grid computation migration, the range resulting differences was minimized as much as possible (Fig. 9). A majority (75%) of the differences were within ±20 W m−2, with only a few locations exceeding 100 W m−2 (Fig. 10). The rigorous benchmarking effort described above was essential for ensuring that existing capabilities in the AGRMET system can be effectively replicated with LIS, in a more computationally scalable manner. Establishing this benchmarking exercise also served as a baseline for all future enhancements included in the 557 WW environment.
Comparisons of the LIS and AGRMET computed products valid at 2100 UTC 21 Dec 2005. (a) The sensible heat flux differences, (b) latent heat flux differences, (c) 0–10-cm soil moisture differences (kg m−2), and (d) 0–10-cm soil temperature differences (K).
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
Histogram of the differences of the LIS–AGRMET net radiation computations during benchmarking evaluation, on 28 Feb 2006. An overwhelming majority of the computations were within ±20 W m−2.
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
Intercomparison evaluations for randomly chosen grid points were performed, with sites locations on the North American (Fig. 11), Asia (Fig. 12), and Africa (Fig. 13) each representing different continents and climatic regimes. For each location, the AGRMET and LIS-AGRMET net radiation, 0–10-cm soil temperature, surface air temperature, and 0–10-cm soil moisture results were plotted and compared for the entire testing period from 2 December 2005 to 28 February 2006. For each point comparison, the LIS and AGRMET output was in general agreement, with some minor differences noted. For Washington D.C., the radiation and average surface temperature forcings were nearly identical during the evaluation period with some minor differences in the computed soil temperature and soil moisture.
LIS (red line) and AGRMET (black line) point intercomparisons for a North America continental location for the period of 2 Dec 2005–28 Feb 2006. The intercomparisons plots include (a) net radiation, (b) 0–10-cm soil temperature, (c) surface temperature, and (d) 0–10-cm soil moisture.
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
LIS (red line) and AGRMET (black line) point intercomparisons for an Asian continent location for the period of 2 Dec 2005–28 Feb 2006. The intercomparisons plots include (a) net radiation, (b) 0–10-cm soil temperature, (c) surface temperature, and (d) 0–10-cm soil moisture.
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
LIS (red line) and AGRMET (black line) point intercomparisons for northern Africa for the period of 2 Dec 2005–28 Feb 2006. The intercomparisons plots include (a) net radiation, (b) 0–10-cm soil temperature, (c) surface temperature, and (d) 0–10-cm soil moisture.
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
6. Implementation of the LIS-AGRMET system at AFWA
The initial software development and integration project duration lasted approximately one year until the initial LIS-AGRMET system was delivered, after which there was an extended period of software evaluation of the LIS software on USAF computing systems. During this time, the USAF and NASA established a joint, on-site support position at the 557 WW as part of the longer-term strategic partnering to foster interagency collaboration and directly support the testing and integration of LIS software. During the testing period, a full evaluation of the LIS software was conducted ensuring the forcing datasets computed in the LIS software were benchmarked against the AGRMET-computed forcing as the baseline and comparing the resulting products (Fig. 14).
Volumetric soil moisture (m3 m−3) results for (a) AGRMET compared to the (b) LIS-AGRMET valid 30 Jan 2006, from initial testing after integration onto the USAF development system. The AGRMET data from the 3-hourly results are projected onto a lat–lon domain centered on the date line and contains results over the Antarctic. The LIS results projection is centered over the prime meridian and the southern extent of the domain stops at 60°S latitude.
Citation: Weather and Forecasting 37, 12; 10.1175/WAF-D-22-0064.1
Testing the LIS software within the USAF computing environment revealed minor issues requiring software modifications related to the computing environment specific configurations, a minor precipitation gauge observation processing issue, and to enable the LIS-AGRMET and AGRMET systems to run in parallel for an extended period of time to support downstream end-user testing and integration of the data. To prepare for operational implementation within a USAF production environment, further testing and modification of the software was needed in order to ensure it would operate as expected, handle data outages from some of the input data, and execute reliably without unexpected errors. Changes to the source code included minor administrative changes, including extending character arrays holding path and filename information, making additional changes to error handling messaging, and developing the software scripts that preprocess the environment, and pull observations from appropriate databases. The testing and software checkout occurred over a 2-yr period simulating operations on a parallel server. There were additional internal checks of the LIS-AGRMET results against the parallel AGRMET output to ensure the results were comparable. Once the software was thoroughly checked out, planning for implementation began in 2008. Finally, an important component of the testing was to ensure any changes made to support operations were also integrated back into the LIS baseline at NASA to ensure a common baseline for future deliveries. This dual R2O and operations-to-research common software pathway facilitates more rapid deliveries of future model updates, and improves operational understanding of operational requirements for model developers at research institutions. The LIS system achieved initial operational configuration (IOC) in the AFWA production environment on 24 February 2009. The implementation was fully successful, and the modeling systems software stability has proven to be very reliable with very few operational errors and downtime not caused by external issues (system downtime, etc). The IOC is defined as a period of dual operations where both the LIS-AGRMET and legacy AGRMET software were running in parallel in order to allow downstream users to switch over to the new products. Additionally, during times of IOC the older AGRMET software was frozen with no new science or significant software updates outside those needed to maintain operations.
7. Links to decision support systems
The AGRMET and later LIS-AGRMET systems are datasets used within the U.S. Department of Agriculture Global Crop Production System. Prior to 2010, the AGRMET and LIS-AGRMET results and software were only available as a limited distribution product, including the AGRMET modules within the LIS software baseline, in order to limit the impact on global commodities markets since the AGRMET products were used within the USDA decision-making process. The USDA relied on the global datasets available within the AGRMET system to support assessments of agro-meteorological conditions and quantitative regional agricultural yield and conditions computations and was concerned about the impacts on the commodity trading markets should the AGRMET data be released prior to the public release of the commodity production estimates. The expansion of satellite observations of vegetation health has reduced the reliance on the AGRMET output, making the products more readily distributable. However, the configurability of LIS and higher resolution capabilities enabled a much broader capacity to support a number of additional security-related decisions.
The U.S. Army Engineer Research and Development Center (ERDC) has been investigating linking the LIS output to a global soil strength and vehicle mobility analysis tool (Bieszczad et al. 2016; Ueckermann et al. 2018; Audette et al. 2017), supporting trafficability decisions. Further, the LIS-AGRMET capabilities have been used in decision support to understand the social and economic impacts of drought on society, focused on the horn of Africa. Roningen and Eylander (2014) specifically looked at using environmental parameters from LIS-based model runs linked to human displacement in extreme drought conditions in Somalia during the 2011 drought. Sufficient data exists to suggest that climate played a role in regional insecurity in the region during the 2011 drought, though that link may have been indirect. This and similar studies contributed to a NASA-lead LIS-based (Arsenault et al. 2020) study similarly looking to predict seasonal and subseasonal changes in drought conditions with a tie to regional stability. The ability of such large environmental modeling systems to directly link with downstream decision support applications, including those linked with social and economic impacts, is increasing with the growing capabilities to observe and understand environmental conditions more completely, and predict seasonal and subseasonal trends and link those trends with social and economic impacts.
8. Summary and takeaways
The LIS-AGRMET system was implemented in 2009 and is now the primary global land data assimilation system supporting the USAF weather analysis and prediction system. The partnership between the USAF and NASA enabled a two-way technology transition of software and science, supported the integration of AGRMET components into the LIS framework with subsequent transition of the LIS-AGRMET software into the USAF operational environment. The LIS software was implemented at ¼° global resolution with parallel software operation enabling a more efficient computing process, with both configurable resolution and domain configuration to support regional, mesoscale model initialization, while also enabling a retrospective processing capability that supported historical model evaluations offline much more efficiently than the existing AGRMET software. However, the benefits to both organizations extended beyond the software improvements into providing considerable organizational benefits, providing NASA a key pathway to transition community supported research and development into USAF operations through a common software baseline. The USAF benefits from community supported science and the shared software baseline provides for rapid technology transition significantly decreasing the time to transition new science from years to months or less. USAF investments into the LIS system feeds back to NASA, providing a more advanced modeling capability that can be used for research and development. Further, since much of the software baseline is released as open source, the broader science community also benefits from these investments. The result is a key, mature enabling technology that is well documented both technically and scientifically, provides a solid bridge between research and operations communities, and has become a top global community technical capability in land data assimilation technology.
The partnership is supported by a joint testbed on the DoD HPCMP that enables both NASA and USAF, and other DoD partners, to share data and collaboratively test improvements to the LIS, and provides the facility to establish a long-term meteorological forcing archive to support extensive and extended LIS testing. The LIS testbed on the HPCMP contains USAF-specific inputs to execute the LIS-AGRMET system in a similar manner to an operational configuration, with all the gridded and individual observations archived starting from 1 December 2005 through present. This collaborative joint software development environment contributed to the successful LIS IOC achieved in February 2009. The flexible software testbed enabled greater flexibility to collaborate and support executing LIS simulations using similar or nearly identical model configuration to support the benchmarking process. This provides the ability to understand both scientific and software performance changes for any model physics update or model configuration change, and provides not only the model results, but also the impacts that a configuration change may have on model run time, memory usage, processing requirements or the impacts of model resolution, projection and grid may have on the results as compared to existing operational configurations. These details may be scientifically trivial, but often pose significant hurdles to successful transition efforts. Similarly, testing software on the same or similar computing equipment is important in order to test model configurations with the same software compilers, hardware computing and memory configurations, since this provides an important baseline. Porting of software and libraries are often time consuming and are another significant barrier. While the broader research community often maintains large computation testbeds to support their research missions, the joint DoD-NASA Joint LIS testbed is a key capability that contributed to the rapid testing of the model, benchmarking LIS-AGRMET performance against AGRMET operational results, and better understanding of computing requirements for new components integrated into the LIS baseline prior to implementation. The use of the testbed continues to facilitate rapid technical transition of LIS updates into operations.
Further, the partnership between the USAF and NASA has continued beyond the initial LIS-AGRMET IOC, facilitating a technology transition of a number of improvements into the AFWA/557WW production environment. Once the software was being prepared for implementation at the AFWA/557WW, the USAF and NASA developed a long-term strategic research investment plan aimed at continuing to improve many of the LIS-AGRMET components, including the global precipitation analysis, snow depth analysis, and data assimilation capabilities. The initial development plan led to several additional incremental development projects, including a study of several satellite-based global precipitation analyses (Tian et al. 2009), and the integration of the NOAA Climate Prediction Center Morphing (CMORPH; Joyce et al. 2004) global precipitation analysis system as a replacement for GEOPRECIP in 2013. The integration of the ensemble Kalman filter into LIS was sponsored by the USAF in 2008 (Kumar et al. 2008b; Reichle et al. 2010) and facilitated a number of investigations and implementation of the assimilation of remotely sensed soil moisture, snow cover, and snow water equivalent estimates. Further, the evaluation and improvements to the USAF global snow analysis were supported within this program, including the development of the AFWA-NASA Snow Algorithm (Foster et al. 2011; Hall et al. 2010) and contributed to snow assimilation and investigations to better incorporate the snow analysis capability within the LIS framework (Kumar et al. 2013; Liu et al. 2013).
The partnership between the USAF and NASA also more directly links the research community with those making environmentally informed decisions. Weather agencies within the USAF, as a key supplier of weather and climate information to operational USAF units, the U.S. Army and a number of U.S. government organizations, are also key partners in the development of weather analysis and prediction technologies with U.S. civilian agencies. The USAF-NASA partnership, through LIS, reduces the time needed to integrate new science to better support end users and creates a strong link to decision support.
The integration of LIS into operations, its capacity to both use remotely sensed observations, execute at much higher resolutions, and perform parallel computations were key considerations driving to establish the partnership and supported the final decision to integrate and operationalize the modeling software. The partnership benefits both organizations beyond the shared software baseline; it provides a strong pathway to transition both NASA software and satellite data into USAF operations, which not only supports the users of USAF products, but provides NASA with both a return on investment for organization research and development investments, as well as finding a partner willing to provide strategic investments to mature and transition the technology.
Acknowledgments.
Funding for NASA participation in this effort was sponsored by the USAF Weather Agency Weather and Forecasting Improvements Program with an initial award of NVSHAX-5680448. Further awards from the Weather and Forecasting Improvements Program supported continued testing and integration of the software and system during the testing period. The evaluation and integration of the Land Information System software was conducted using staff personnel from the Air Force Weather Agency, now the 557th Weather Wing. Funding for the development of this paper was provided by the U.S. Army Engineer Research and Development Center as part of a doctoral training and sabbatical sponsorship program. Finally, the USAF and NASA team would like to extend appreciation to Dr. Kenneth Mitchell, formerly of NCEP/EMC, who directly and indirectly contributed to the development, and subsequent improvements to the AGRMET modeling system, provided significant leadership to the formation of the partnership resulting in the Noah LSM, and was key to introducing the group of individuals listed as authors on this paper that resulted in the development. Dr. Mitchell’s mentoring and dedication to fostering a highly collaborative, cross-agency partnership early in the LIS development directly contributed to the success of the program. Finally, we thank Dr. Michael Ek and two anonymous reviewers for their comments and suggestions that helped to strengthen this manuscript.
Data availability statement.
This paper describes the historical development of the NASA Land Information System software, as a research, development, and integration activity that occurred over a decade ago as a sponsored project within the USAF Weather and Forecasting Improvement program. However, the data used in the analysis section of this paper are no longer complete and were only partially available at this time. Historical USAF AGRMET production output is available in GRIB format and archived on NASA Goddard Space Flight Center supercomputing systems for technical analysis. The initial archived LIS results were based on LIS version 5.0 testing and comparisons to AGRMET production output were lost during an archive system migration. The LIS configuration has since undergone significant enhancements and the data from the operational system is available through subscription services with the 557WW to approved organizations. Note also that the LIS software is released publicly through the LIS source code repository on Github (https://github.com/NASA-LIS/LISF) and available to any registered user to download and view.
APPENDIX
List of Acronyms
557WW |
U.S. Air Force 557th Weather Wing |
AFW |
Air Force Weather |
AFWA |
Air Force Weather Agency |
AFGWC |
Air Force Global Weather Central |
AGRMET |
Agriculture Meteorology Model |
Agromet |
Agriculture Meteorology Model |
CDFSII |
Cloud Depiction and Forecast System Version II |
CMORPH |
Climate Prediction Center Morphing Technique |
DMSP |
Defense Meteorological Satellite Program |
DOD |
Department of Defense |
DSRC |
Distributed Supercomputer Resource Center |
EMC |
Environmental Modeling Center |
ERDC |
Engineer Research and Development Center |
ETAC |
Environmental Technical Applications Center |
GDAS |
Global Data Assimilation System |
GEOPRECIP |
Geostationary Infrared Precipitation Analysis Model |
GFS |
Global Forecast System |
GRIB |
Gridded binary |
HP |
Hewlett Packard Company |
HPCMP |
DOD High Performance Computing Modernization Program |
IBM |
International Business Machines |
IOC |
Initial operating configuration |
LDAS |
Land Data Assimilation System |
LIS |
Land Information System |
LSM |
Land surface model |
MM5 |
Mesoscale Model version 5 |
MODIS |
Moderate Resolution Imaging Spectroradiometer |
NASA |
National Aeronautics and Space Administration |
NCAR |
National Center for Atmospheric Research |
NCEP |
National Centers for Environmental Prediction |
NetCDF |
Network Common Data Form |
NOAA |
National Oceanic and Atmospheric Administration |
NOGAPS |
Navy Operational Global Atmospheric Prediction System |
NWP |
Numerical weather prediction |
NWS |
National Weather Service |
OSU |
Oregon State University |
R2O |
Research to operations |
SNODEP |
Snow Depth Analysis Model |
SRTM |
Shuttle Radar Topography Mission |
SSM/I |
Special Sensor Microwave/Imager |
SSM/IS |
Special Sensor Microwave Imager/Sounder |
STATSGO |
State Soil Geographic Database |
USAF |
U. S. Air Force |
VIIRS |
Visible Infrared Imaging Radiometer Suite |
WRF |
Weather Research and Forecasting Model |
WWMCA |
World Wide Merged Cloud Analysis |
REFERENCES
Arsenault, K. R., and Coauthors, 2020: The NASA hydrological forecast system for food and water security applications. Bull. Amer. Meteor. Soc., 101, E1007–E1025, https://doi.org/10.1175/BAMS-D-18-0264.1.
Audette, W., M. P. Ueckermann, C. A. Brooks, D. R. Callendar, J. D. Walthour, and J. Bieszczad, 2017: Benchmarking DASSP, a cloud-based downscaling system for 30-meter global soil moisture estimates. 31st Conf. on Hydrology, Seattle, WA, Amer. Meteor. Soc., 11.1, https://ams.confex.com/ams/97Annual/webprogram/Paper307974.html.
Barnes, S. L., 1964: A technique for maximizing details in numerical weather map analysis. J. Appl. Meteor., 3, 396–409, https://doi.org/10.1175/1520-0450(1964)003<0396:ATFMDI>2.0.CO;2.
Bieszczad, J., M. P. Ueckermann, C. A. Brooks, R. Chambers, W. E. Audette, and J. D. Walthour, 2016: DASSP: A system for high-resolution, global prediction of soil moisture content and soil strength. 30th Conf. on Hydrology, New Orleans, LA, Amer. Meteor. Soc., 77, https://ams.confex.com/ams/96Annual/webprogram/Paper288342.html.
Chen, F., and J. Dudhia, 2001: Coupling an advanced land surface-hydrology model with the Penn State–NCAR MM5 modeling system. Part I: Model implementation and sensitivity. Mon. Wea. Rev., 129, 569–585, https://doi.org/10.1175/1520-0493(2001)129<0569:CAALSH>2.0.CO;2.
Chen, F., and Coauthors, 1996: Modeling of land surface evaporation by four schemes and comparison with FIFE observations. J. Geophys. Res., 101, 7251–7268, https://doi.org/10.1029/95JD02165.
Cochrane, M. A., Jr., 1981: Soil moisture and AGROMET models. USAFETAC Tech. Note USAFETAC/TN-81/001, 30 pp.
d’Entremont, R. P., and G. B. Gustafson, 2003: Analysis of geostationary satellite imagery using a temporal differencing technique. Earth Interact., 7, https://doi.org/10.1175/1087-3562(2003)007<0001:AOGSIU>2.0.CO;2.
d’Entremont, R. P., R. Lynch, G. Uymin, J. Moncet, R. B. Aschbrenner, M. Conner, and G. B. Gustafson, 2016: Application of optimal spectral sampling for a real-time global cloud analysis model. Wea. Forecasting, 31, 743–761, https://doi.org/10.1175/WAF-D-15-0077.1.
Ek, M. B., K. E. Mitchell, Y. Lin, E. Rogers, P. Grunmann, V. Koren, G. Gayno, and J. D. Tarpley, 2003: Implementation of Noah land surface model advances in the National Centers for Environmental Prediction operational mesoscale Eta model. J. Geophys. Res., 108, 8851, https://doi.org/10.1029/2002JD003296.
Eylander, J. B., 2013: Land Information System (LIS) development plan: FY13-FY17. ERDC/CRREL Tech. Note TN-13-2, Hanover, NH, 40 pp.
Eylander, J. B., D. M. Rozema, and J. K. Lee, 2007: Development Plan for the Air Force Weather Agency (AFWA) Land Information System (LIS). USAF Weather Agency Strategic Plans and Programs Directorate, Offutt AFB, NE, 26 pp.
Foster, J. L., and Coauthors, 2011: A blended global snow product using visible, passive microwave and scatterometer satellite data. Int. J. Remote Sens., 32, 1371–1395, https://doi.org/10.1080/01431160903548013.
Gayno, G., and J. Wegiel, 2000: Incorporating global real-time surface fields into MM5 at the Air Force Weather Agency. 10th Penn State/NCAR MM5 Users’ Workshop, Boulder, CO, National Center for Atmospheric Research, 62–65.
Gustafson, G. B., and R. P. d’Entremont, 2000: Development and validation of improved techniques for cloud property retrieval from environmental satellites. USAF Research Lab Tech. Rep. AFRL-VS-TR-2001-1549, Air Force Research Laboratory Space Vehicles Directorate, Hanscom AFB, MA, 48 pp.
Gutman, G., and A. Ignatov, 1998: The derivation of the green vegetation fraction from NOAA/AVHRR data for use in numerical weather prediction models. Int. J. Remote Sens., 19, 1533–1543, https://doi.org/10.1080/014311698215333.
Hall, D. K., G. A. Riggs, J. L. Foster, and S. V. Kumar, 2010: Development and evaluation of a cloud-gap-filled MODIS daily snow-cover product. Remote Sens. Environ., 114, 496–503, https://doi.org/10.1016/j.rse.2009.10.007.
Hall, S. J., 1986: AFGWC snow analysis model. Offutt AFB, NE, 23 pp.
Hoke, J. E., C. J. L. Hayes, and L. G. Renninger, 1985: Map projections and grid systems for meteorological applications. Offutt AFB, NE, 87 pp.
HQ AFWA, 2005: Algorithm description for the cloud depiction and forecast system II. Offutt AFB, NE, 365 pp.
Idso, S. B., 1981: A set of equations for full spectrum and 8‐ to 14‐μm and 10.5‐ to 12.5‐μm thermal radiation from cloudless skies. Water Resour. Res, 17, 295–304, https://doi.org/10.1029/WR017i002p00295.
Joyce, R. J., J. E. Janowiak, P. A. Arkin, and P. Xie, 2004: CMORPH: A method that produces global precipitation estimates from passive microwave and infrared data at high spatial and temporal resolution. J. Hydrometeor., 5, 487–503, https://doi.org/10.1175/1525-7541(2004)005<0487:CAMTPG>2.0.CO;2.
Kopp, T. J., 1995: The Air Force Global Weather Central Surface Temperature Model. Offutt AFB, NE, 30 pp.
Kumar, S. V., and Coauthors, 2006: Land information system: An interoperable framework for high resolution land surface modeling. Environ. Modell. Software, 21, 1402–1415, https://doi.org/10.1016/j.envsoft.2005.07.004.
Kumar, S. V., C. Peters-Lidard, Y. Tian, R. Reichle, J. Geiger, C. Alonge, J. Eylander, and P. Houser, 2008a: An integrated hydrologic modeling and data assimilation framework. Computer, 41, 52–59, https://doi.org/10.1109/MC.2008.475.
Kumar, S. V., R. H. Reichle, C. D. Peters-Lidard, R. D. Koster, X. Zhan, W. T. Crow, J. B. Eylander, and P. R. Houser, 2008b: A land surface data assimilation framework using the land information system: Description and applications. Adv. Water Resour., 31, 1419–1432, https://doi.org/10.1016/j.advwatres.2008.01.013.
Kumar, S. V., C. D. Peters-Lidard, D. Mocko, and Y. Tian, 2013: Multiscale evaluation of the improvements in surface snow simulation through terrain adjustments to radiation. J. Hydrometeor., 14, 220–232, https://doi.org/10.1175/JHM-D-12-046.1.
Liu, Y., C. D. Peters-Lidard, S. Kumar, J. L. Foster, M. Shaw, Y. Tian, and G. M. Fall, 2013: Assimilating satellite-based snow depth and snow cover products for improving snow predictions in Alaska. Adv. Water Resour., 54, 208–227, https://doi.org/10.1016/j.advwatres.2013.02.005.
Mahrt, L., and M. Ek, 1984: The influence of atmospheric stability on potential evaporation. J. Climate Appl. Meteor., 23, 222–234, https://doi.org/10.1175/1520-0450(1984)023<0222:TIOASO>2.0.CO;2.
Mahrt, L., and H. Pan, 1984: A two-layer model of soil hydrology. Bound.-Layer Meteor., 29, 1–20, https://doi.org/10.1007/BF00119116.
Matthews, E., 1983: Global vegetation and land use: New high-resolution data bases for climate studies. J. Climate Appl. Meteor., 22, 474–487, https://doi.org/10.1175/1520-0450(1983)022<0474:GVALUN>2.0.CO;2.
Miller, D. A., and R. A. White, 1998: A conterminous U.S. multilayer soil characteristics dataset for regional climate and hydrology modeling. Earth Interact., 2, https://doi.org/10.1175/1087-3562(1998)002<0002:CUSMS>2.0.CO;2.
Moore, B. A., S. E. Bertone, K. E. Mitchell, P. B. Rice, and R. D. Neill, 1991: A worldwide near-real time diagnostic agrometeorological model. Preprints, 20th Conf. on Agricultural and Forest Meteorology, Salt Lake City, UT, Amer. Meteor. Soc., 7–11.
Peters-Lidard, C. D., and Coauthors, 2007: High-performance Earth system modeling with NASA/GSFC’s land information system. Innovations Syst. Software Eng., 3, 157–165, https://doi.org/10.1007/s11334-007-0028-x.
Reichle, R. H., S. V. Kumar, S. P. P. Mahanama, R. D. Koster, and Q. Liu, 2010: Assimilation of satellite-derived skin temperature observations into land surface models. J. Hydrometeor., 11, 1103–1122, https://doi.org/10.1175/2010JHM1262.1.
Reynolds, C. A., T. J. Jackson, and W. J. Rawls, 2000: Estimating soil water-holding capacities by linking the Food and Agriculture Organization soil map of the world with global pedon databases and continuous pedotransfer functions. Water Resour. Res., 36, 3653–3662, https://doi.org/10.1029/2000WR900130.
Robinson, D. A., and G. Kukla, 1985: Maximum surface albedo of seasonally snow-covered lands in the Northern Hemisphere. J. Appl. Meteor. Climatol., 24, 402–411, https://doi.org/10.1175/1520-0450(1985)024<0402:MSAOSS>2.0.CO;2.
Roningen, J. M., and J. B. Eylander, 2014: Socio-economic Effects of Drought in the Horn of Africa: Population Movements, Livelihoods, Market Prices, and Infrastructure. Hanover, NH, 120 pp.
Shapiro, R., 1987: A simple model for the calculation of the flux of direct and diffuse solar radiation through the atmosphere. Tech. Memo. AFGL-TR-87-0200, Air Force Geophysics Lab, Hanscom AFB, MA.
Sturm, W. J., 1977: Soil moisture agrometeorological services. USAFETAC Tech. Note USAFETAC/TN-77-3, 37 pp., https://apps.dtic.mil/dtic/tr/fulltext/u2/a321340.pdf.
Tian, Y., and Coauthors, 2009: Component analysis of errors in satellite-based precipitation estimates. J. Geophys. Res., 114, D24101, https://doi.org/10.1029/2009JD011949.
Ueckermann, M. P., J. Bieszczad, and D. R. Callender, 2018: A RESTful API for Python- based server-side analysis of high-resolution soil moisture downscaling data. Eighth Symp. on Advances in Modeling and Analysis Using Python, Austin, TX, Amer. Meteor. Soc., 4.2, https://ams.confex.com/ams/98Annual/webprogram/Paper332957.html.
Vicente, G. A., R. A. Scofield, and W. P. Menzel, 1998: The operational GOES infrared rainfall estimation technique. Bull. Amer. Meteor. Soc., 79, 1883–1898, https://doi.org/10.1175/1520-0477(1998)079<1883:TOGIRE>2.0.CO;2.
Wachtmann, R., 1975: Expansion of atmospheric temperature–moisture profiles in empirical orthogonal functions for remote sensing applications. Preprints, Topical Meeting on Optical Remote Sensing of the Atmosphere, Anaheim, CA, Optical Society of America.
Zobler, I., 1986: A world soil file for global climate modeling. NASA Tech. Memo. 87802.