1. Introduction
a. Assimilation of AIRS radiances: Stating the problems
Data from the Atmospheric Infrared Sounder (AIRS), on board the NASA Aqua satellite, have been extensively used by operational weather forecasting centers worldwide for more than a decade. Evidence that the assimilation of AIRS radiances brought a positive impact on the European Centre for Medium-Range Weather Forecasts operational system was provided, among others, by McNally et al. (2006). Similarly, positive impact was shown for the National Centers for Environmental Prediction (NCEP) system (e.g., Chahine et al. 2006; Le Marshall et al. 2006). Over the following years, other hyperspectral instruments have been designed and placed into orbit: the Cross-track Infrared Sounder (CrIS) on board the Suomi National Polar-Orbiting Partnership (SNPP), launched on 28 October 2011, and the two Infrared Atmospheric Sounding Interferometers (IASIs), on board the MetOp-A and MetOp-B satellites, launched on 19 October 2006 and 17 September 2012, respectively. The combined impact of infrared instruments appears to be dominant in current operational weather forecasting as shown, among others, by Joo et al. (2013) in their evaluation of IASI (MetOp-A) and AIRS in the Met Office system through adjoint methodologies. However, because the number of observations assimilated operationally per cycle is currently approaching 107, it is becoming increasingly difficult for a single instrument to positively impact the forecast skill.
Aggressive data reduction algorithms are necessary because of the computational cost but also for another, less immediate, reason: to reduce the effects of horizontally correlated errors. High data density, while intuitively desirable, increases the possibility of not satisfying the crucial assumption of errors being independent, which is a basic requirement in current operational data assimilation systems. Ochotta et al. (2005) clearly state that “a high spatial and/or temporal data density can severely violate the assumption of independent observation errors.” Since the control of error correlation between observations is an extraordinarily difficult problem, the large volume of AIRS data and other infrared instruments is often suboptimally sampled on a regularly spaced grid through the thinning procedure. This is in apparent contrast with the goals of instrument and algorithm development teams, who are often focused on designing better products in terms of a larger channel selection, larger data volume, and denser coverage.
In addition to suboptimal sampling, another fundamental limitation hinders the full exploitation of AIRS and other infrared sensors: the limitation to the use of clear-sky radiances. Currently, only data from channels that are thought to be unaffected by clouds are assimilated. This is in contrast with the routine operational use of cloud-affected microwave radiances (e.g., Bauer et al. 2010; Zhu et al. 2016) and poses a serious limitation to the usefulness of the infrared data, since clear areas are meteorologically less active. From the point of view of tropical cyclone (TC) forecasting, the rejection of cloud-contaminated channels often leaves evident data-void areas, which are in marked correspondence to tropical cyclone circulations. This is particularly harmful in cases in which no additional observations, such as the ones obtained by hurricane hunter flights, are available inside the storm’s circulation (e.g., Reale et al. 2009b).
b. The potential of adaptive thinning
A reasonable agreement exists on the fact that assimilation of additional observations in regions where the observational network is dense, initial conditions are accurately known, or where error growth is small can yield only modest or no forecast improvements (e.g., Morss et al. 2001). That study, among others, convincingly argued that adaptive strategies, based on ingesting additional observations from areas where observational errors are large, or where forecast errors grow more rapidly, could be very effective. A vast number of adaptive strategies have been proposed, such as techniques based on ensemble spread (e.g., Lorenz and Emanuel 1998), singular vectors (e.g., Gelaro et al. 1999), and ensemble transform techniques (e.g., Bishop and Toth 1999).
From the point of view of the present study, which is focused on the assimilation of infrared radiances, particularly relevant is a seminal work by Liu and Rabier (2003, hereafter LR03) that discusses the potential of high-density observations within an observing system simulation experiment (OSSE) four-dimensional variational data assimilation (4DVar) context. The major finding, confirming their previous, more theoretical, one-dimensional assessment (Liu and Rabier 2002), is that increased observation density with uncorrelated errors always increases the analysis accuracy. At the same time, LR03 found that an increase in observation density degrades both the analysis and forecast if the error correlation between adjacent observations is greater than a certain threshold. Therefore, a separation between an optimal analysis, in which errors are uncorrelated, and a suboptimal analysis, in which errors are assumed to be uncorrelated while in reality they are, could be ideally achievable, based on an error correlation threshold.
Several methodologies, based on the conceptual understanding provided by LR03, were proposed in the following years. Among them, Ochotta et al. (2005) presented two thinning algorithms, called top-down clustering and estimation error analysis, to reduce the number of assimilated observations while retaining the essential information content of the data. This results in an observation density that is greater in rapidly changing regions. In Ochotta et al. (2007), a further improvement and revisitation of the estimation error analysis technique was provided. Bondarenko et al. (2007) presented a comprehensive discussion on the impact of observation density and clarified the fundamental fact that a suboptimal analysis is often a necessary choice because of the difficulty and cost of determining observation error correlations. Lazarus et al. (2010) provided an in-depth comparison of “standard” (i.e., regularly gridded) against “intelligent” (i.e., adaptive) thinning techniques, stating that “simple thinning tends to perform better over the relatively uninteresting homogeneous data regions.” Unfortunately, in spite of the deep underlying theoretical understanding and by the very convincing findings of the above-referred studies, the difficulties associated with the widespread operational implementation of adaptive methods have proven, so far, insurmountable.
Therefore, in practice, most operational centers are still compelled to use a regularly spaced grid to perform data thinning, simply assuming that errors are uncorrelated, even if they are not. The Gridpoint Statistical Interpolation analysis system (GSI; Wu et al. 2002; Purser et al. 2003a, 2003b), which is the foundation of the GEOS Data Assimilation System (DAS), is no exception. Satellite radiance data, including AIRS data, which represent the largest volume of data ingested into the numerical weather prediction system (Rienecker et al. 2008), necessitate subsetting the data through a thinning routine prior to assimilation. The subselection of channels affects the vertical data density and is not the goal of this article: in the experiments described here we have assimilated 117 channels, a choice very similar to NCEP at that time and the Modern-Era Reanalysis for Research and Applications, version 2 (MERRA-2; Gelaro et al. 2017), with the actual channel selection shown by McCarty et al. (2016). The thinning strategy, which is the focus of this article, affects the horizontal data density, is prescribed to a thinning mesh for each instrument type, and is designed to give preferences to observations likely to pass quality control (Rienecker et al. 2008).
AIRS clear-sky radiance data are thinned in observation space, and the thinning strategy is independent of information from the forecast model. For the reference experiment described in this study, and used in routine operations at the NASA Global Modeling and Assimilation Office (GMAO) from August 2014 to May 2015, thinning is performed on a uniformly spaced grid of 145 km × 145 km. Within this grid, multiple instrument fields of view are present from which suitable data are chosen to be passed to the quality control routines. This selection is performed by assigning a score to each radiance that is produced from a weighted combination of individual scores from various suitability tests. The scoring system is designed to prefer data that minimize the distance from the center of the grid box and temporal departures from the analysis time, are cloud-free, and are present over water. The cloud-free criterion is evaluated using a window channel as a reference, with a higher score assigned to warmer brightness temperatures, indicating that the signal is more likely to reach the surface rather than detecting a cloud layer. Evaluation of AIRS radiance data includes additional scoring to reduce contamination from thin cirrus.
After thinning, the selected radiance data are evaluated for quality control by the GSI using comparisons to estimates from the forecast model and nearby observations for detection of poor quality data and systemic biases that may necessitate correction before assimilation (Derber and Wu 1998; Rienecker et al. 2008). Quality control procedures may reject a given radiance due to the presence of clouds or precipitation, uncertainty in the surface emissivity estimate, or the elimination of outliers produced by gross error check. Biases may result from errors in the satellite instrument, the radiative transfer model, or the background atmospheric state produced by the forecast model.
c. The potential of cloud-cleared radiances
The use of AIRS (and other infrared instruments) has been generally restricted to the assimilation of clear-sky radiances, which considers only channels insensitive to clouds, either within a clear field of view (FOV) or with a sensitivity limited to above the cloud top. The obvious limitations stemming from the use of clear-sky data were discussed extensively, among several others, by Reale et al. (2008), Reale et al. (2009a,b), and Reale et al. (2012). In these works, the assimilation of AIRS retrievals obtained under partly cloudy conditions, also known as cloud-cleared retrievals (e.g., Susskind et al. 2006), was compared to the assimilation of clear-sky radiances, which was and continues to be the dominant operational approach worldwide. Results showed that assimilation of cloud-cleared retrievals improved 1) midlatitude weather systems (Reale et al. 2008) via improvement of the lower-tropospheric temperature structure in polar regions, probably because of a better representation of low-level stratus clouds; 2) tropical cyclogenetic processes in the Atlantic (Reale et al. 2009a); 3) tropical cyclones in the north Indian Ocean (Reale et al. 2009b); 4) tropical cyclone–produced precipitation worldwide (Zhou et al. 2010); and 5) large-scale flood-producing precipitation (Reale et al. 2012), with a focus on the forecasting of a major flood-producing precipitation event that occurred in Pakistan in 2010. The beneficial changes in the tropical regions were attributed to the assimilation of upper-tropospheric temperature information in areas that are meteorologically very active, particularly around TCs or large convective systems, in which clouds are present. For example, the assimilation of AIRS-derived information in cloudy regions could create a small upper-tropospheric temperature contrast between the area affected by a TC circulation (becoming warmer as a consequence of ingestion of AIRS data) and the surrounding environment (becoming slightly cooler), which resulted in a deeper and more confined center pressure. It was therein also argued that clear-sky radiances predominantly come from relatively stagnant regions of the atmosphere, such as large anticyclones, and are likely to exert less impact than data from cloudy regions, which are generally more active.
The aforementioned results obtained by assimilation of retrievals in partially cloudy regions, while informative, have limited the practical value for the forecasting community because of the almost exclusive operational use of direct radiance assimilation instead of retrievals. This choice stems partly from theoretical reasons such as, for example, avoidance of additional sources of background errors (e.g., McCarty et al. 2009). While the cloud-clearing algorithm used in AIRS version 5 retrievals did not require any model background (e.g., Chahine et al. 2006; Le Marshall et al. 2006; Susskind et al. 2006, 2011), additional practical and crucial constraints of real-time forecasting, such as latency, makes the use of retrievals impractical for the operational community. However, the lessons learned from the comparison between cloud-cleared retrievals and clear-sky radiances cannot be ignored. Consistently better results were obtained by assimilating version 5 retrievals, measured with a diverse set of metrics, ranging from global skill to TC intensity and extreme precipitation forecasting. This suggests that improved coverage over meteorologically active regions, which are generally affected by clouds, is an extremely important aspect. However, in spite of the fact that AIRS cloud-cleared radiances based on the methods of Chahine et al. (2006) have been available for a long time at the NASA Distributed Active Archive Center (DAAC), and that other centers have produced similar datasets, very few documented efforts of assimilating AIRS radiances in cloudy regions exist in the literature (e.g., Pangaud et al. 2009; Singh et al. 2011; Wang et al. 2015).
The purpose of this article is twofold. First, it intends to provide some evidence on the benefits that could be brought to operational forecasting by revisiting the overall thinning strategy with a very simple adaptive approach, based on the conceptual understanding provided by the pioneering work by Liu and Rabier (2002, 2003), the comprehensive reviews offered, among others, by Lazarus et al. (2010), and in agreement with the goals sought by operational centers (e.g., Zhu and Boukabara 2015). The second goal of this article is to show the value of AIRS cloud-cleared radiances, which are not currently used in operational centers, but that could seamlessly replace clear-sky radiances. In this regard, the article intends to reconcile the apparently contrasting concerns of operational forecasts, constrained by the problems of error correlation and computational cost, and the goals of the AIRS Science Team, which would advocate an improved use of AIRS products.
The article is organized as follows. Section 2 discusses the model and datasets, section 3 provides a general description of the clear-sky experiments, with section 4 showing the results. Section 5 investigates the fraction of beneficial observations globally and on the tropical cyclone scale with the adjoint of the forecast system, section 6 provides a general description of the cloud-cleared experiments, with section 7 illustrating the corresponding results. Section 8 puts the results into the context of current research, and, finally, section 9 states the conclusions of this work.
2. Model and data assimilation system
All the experiments produced as part of this work were carried out with the NASA Global Earth Observing System data assimilation and forecast system, version 5, which merges the cubed-sphere (Lin 2004) hydrostatic version of the model (Molod et al. 2012), with the GSI analysis system developed by NCEP and modified by the GMAO (Rienecker et al. 2008). Specifically, the version of GEOS used in this study is a frozen version (5.13.0p1) of the system, more recent than the one used to produce the Modern-Era Reanalysis for Research and Applications, version 2 (Gelaro et al. 2017), run with a cubed-sphere geometry of 360 × 360 grid cells (c360) within each of the six faces of the gnomonic cubed-sphere grid (Putman and Lin 2007) nearly uniformly distributed around the globe. This corresponds to a horizontal resolution of about 0.25° × 0.3125° around the equator [40 000 km/(360 × 4 grid cells) ≈ 25 km]. The vertical resolution is 72 hybrid-eta layers extending to 0.01 hPa.
This GEOS version was the last one with a three-dimensional variational data assimilation (3DVar) and was identical to the version used semioperationally in the GMAO until May 2015, except for disabling the vortex relocator. The current GEOS includes a hybrid four-dimensional ensemble–variational assimilation scheme (4DEnVar), but there are several reasons for performing the experiments described in this article within a 3DVar context. Among them, as noted by Morss et al. (2001), ingestion of data is simpler to understand and the computational cost is lower, which allows for the production of a very large number of experiments with increased statistical significance. It should also be noted that it was essential to use the same version of the model for all the experiments, whose production spans a period of more than two years, starting in 2015 and ending in 2017. Finally, this choice of disabling the vortex relocator was made to show how AIRS data alone can constrain the position of a storm.
3. Clear-sky experiments setting
A first set of OSEs is performed to investigate the impact of adaptive thinning on clear-sky radiances. While the focus of this article is on cloud-cleared radiances, the findings obtained by the investigation of clear-sky radiances are a necessary prerequisite. This set consists of a number of parallel data assimilation runs starting from 1 September 2014 and ending on 10 November 2014, each assimilating all the observations used operationally at that time, and differing from one another only in the specific treatment of AIRS data. Seven-day forecasts are initialized at 0000 UTC daily from each analysis produced.
Specifically, the clear-sky OSE set comprises a reference experiment (RAD), which, in addition to data used operationally, assimilates clear-sky radiances thinned through a 145 km × 145 km grid, which is identical to that at NCEP and also to the semioperational version used by GMAO at that time (the current GMAO system has increased the size to 180 km). This is regarded as the control experiment. Then, two extreme perturbation experiments are performed, identical to RAD except for altering the AIRS global data density through much more (or less) aggressive thinning. These extreme thinning perturbations represent “bounding” experiments with a drastically larger or smaller data density: a thinning box of 75 or 300 km is used in the experiments named RAD2 and RAD3, respectively. Since only one radiance is chosen to be assimilated within the box, increasing or decreasing the thinning box size allows less or more data in the analysis. More precisely, RAD2 ingests approximately 4 times more data than RAD, and RAD3 about one-quarter as many.
Aside from the bounding experiments, a simple, adaptive, TC-centered thinning scheme, which combines two different AIRS data densities, is applied to the other experiments. Specifically, the SThin experiment uses two different data densities resulting from thinning boxes: one inside a “domain” activated by the National Hurricane Center (NHC)–Joint Typhoon Warning Center (JTWC) TC best-track dataset (BTD) information (also known as HURDAT2), and surrounding any TC present at a given time, and the other global, outside the TC domain. Specifically, the data distribution in the experiment SThin uses a thinning box of 75 km (as RAD2) in moving domains spanning 15° × 15° each centered on any TC present worldwide, and a second based on a thinning box of 300 km (as RAD3) is used everywhere else. This results in a global density of about one-quarter compared to the control, except around TCs, where the density is 4 times larger than RAD. In short, SThin has the same density as RAD2 inside the TC domains, and the same as RAD3 outside the TC domains. Whenever there is a TC somewhere on the globe, the NHC–JTWC BTD information (which contains, among other data, the TC position) is used to create a domain around the TC in which denser AIRS data coverage is assimilated. During the period in which the experiments take place, there are 23 TCs worldwide, and at least one TC is present for about 90% of the duration of the assimilation. More precisely, the adaptive thinning is used 266 times over 284 six-hourly time steps, and up to five TCs are present simultaneously (17 September 2014) in all basins.
In a real-time setup, the BTD information could be seamlessly replaced with the “TC Vitals” information (Trahan and Sparling 2012). The data densities in the experiments are listed in Table 1 and can be visualized in Fig. 1, which magnifies a region over the western North Atlantic, around Hurricane Gonzalo at 0600 UTC 15 October 2014, for clarity. It is important to note that the size of the TC domain (15° × 15°) in the experiment SThin results from being the best of a number of five experiments. The remaining four are not discussed in this paper but are listed for completeness sake and briefly commented upon. These are named SThin2, SThin3, SThin4, and SThin5. The first two alter the thinning box sizes, and the second two the TC domains. Finally, the experiment named OPS represents the version of the model that was used operationally by the GMAO at the time and as late as 2015. OPS is exactly the same model version as RAD and differs only because the vortex relocator is used in the former. There is also another very minor difference in the experiment setup: the sea surface temperatures (SSTs) used in the assimilation are updated daily at 0000 UTC in RAD and at 1200 UTC in OPS, but this cannot possibly affect the skill. While we refer to RAD as the rigorous control experiment, the additional reference to OPS is very important because it shows that the use of the vortex relocator, thought to be useful for TC purposes, is slightly harmful to the global skill. Moreover, this article will suggest in the following sections that the supposed improvement to the TC analysis caused by the vortex relocator could be obtained instead by adaptive assimilation of AIRS cloud-cleared data.
Thinning experiments setup, clear-sky radiances. A blank row separates OPS, RAD, RAD2, and RAD3 (global thinning experiments) from the adaptive experiments. The first column shows the experiment names; the second column is the size of the thinning box adopted globally; the third column is the size of the TC-centered, moving domain, in which a denser thinning is adopted; the fourth column is the thinning box size inside the TC domain; the fifth column is the density of AIRS radiances assimilated globally compared to the reference x in the RAD experiment; and the sixth column is the density of AIRS radiances assimilated inside the TC domain, compared to RAD. Among the five adaptive thinning experiments, SThin (highlighted in bold) combines the data density of RAD3 and RAD2 and has been determined to be the best of the five. The experiment settings for SThin2, SThin3, SThin4, and SThin5 are listed for completeness. In all of the experiments except OPS, the vortex relocator is disabled so as to give AIRS data the possibility to constrain the storm position.
Clear-sky AIRS radiance coverage in all the experiments at 0600 UTC 15 Oct 2014. Hurricane Gonzalo’s center is marked with a boldface L, from the NHC. Circles indicate locations of assimilated radiances: O-F brightness temperatures (K), channel 169, corresponding to approximately 185 hPa.
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
Figure 1 illustrates the limitations of the clear-sky approach, in which only channels thought to be completely unaffected by clouds are evaluated for the thinning and then assimilated. In fact, at the RAD data density level, a large data-void area can be seen around Hurricane Gonzalo, because all of the channels considered to be “cloud contaminated” are rejected before the thinning. However, the RAD2 data distribution, even if still based upon clear-sky radiances, provides some hint of the hurricane outer structure because of the use of approximately 4 times more radiances. In fact, particularly evident are some banded structures to the north and east of the center. A smaller thinning box means that some observations can be accepted by the cloud-detection algorithm in narrow clear filaments between rainbands. The smaller the box, the higher the likelihood that some radiances in small clear areas can be assimilated. The experiment RAD3 uses a 300-km thinning box, which causes AIRS data to be reduced to approximately one-quarter compared to RAD. In contrast with RAD2, the RAD3 data distribution gives almost no information in the proximity of the hurricane’s center.
4. Clear-sky experiment results
Figure 2 shows the 500-hPa global anomaly correlation for all the experiments whose coverage is shown in Fig. 1, as a function of forecast time. The differences of each experiment with respect to both OPS and RAD are also displayed in the same figure. When the OPS experiment serves as the reference for this plot, the improvements or degradations in skill shown in the middle panel are plotted with respect to the OPS forecast. Similarly, the lower panel allows us to discern improvements or degradations compared to RAD. It is worth comparing against both: in fact, while RAD is the “rigorous” control experiment, OPS represents the model that was used operationally at that time, and one goal of this article is to show that, aside from the large improvements in TC representation (to be shown later), no degradation of global skill occurs as a consequence of the adaptive thinning.
Global anomaly correlation for AIRS clear-sky radiance assimilation experiments, computed for the forecasts initialized from 21 Sep to 31 Oct 2014 (41 7-day forecasts, 500-hPa height). (top) Forecast skill as a function of forecast time, (middle) experiment minus OPS forecast skill difference, and (bottom) experiment minus RAD forecast skill difference. The horizontal bars represent statistical significance.
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
The most striking result evident from Fig. 2 is that the two bounding experiments, RAD2 and RAD3, characterized by the most extreme thinning, produce the worst and best global skill of all the experiments, respectively. The improvement in skill for RAD3 is statistically significant compared to OPS. RAD is significantly better than OPS, indicating that the use of the vortex relocator alone, while profitable from a TC forecasting perspective, may be slightly harmful to the global skill, (since it is reasonable to assume that no impact comes from updating SSTs at 0000 or 1200 UTC, which is, as stated, the only other difference between RAD and OPS). As previously noted, SThin is chosen as the best among a number of adaptive thinning configurations listed in Table 1. It is worth noting that all have more global skill than the operational GEOS of that time, but slightly less than SThin (not shown).
The response to the two bounding experiments is also captured by other metrics. For example, Fig. 3 shows the root-mean-square error (RMSE) computed over the tropics for temperature in the 24- and 120-h forecasts. RAD3 produces a small improvement and RAD2 a large deterioration compared to RAD and OPS in the 24-h forecasts. The effect is maintained, even if smaller, up to day 5.
Vertical profiles of temperature RMSE for AIRS clear-sky radiance assimilation experiments: (left) 24-h forecast; (right) 120-h forecast. Results are for the tropics only (30°S–30°N). All the RMSE profiles for experiments with adaptive thinning are contained between RAD2 and RAD3 (not shown for clarity).
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
It is possible that the surprising beneficial effect of more aggressive data thinning for clear-sky AIRS radiances can be explained in terms of error correlation. Following LR03, it would seem that error correlation, for the AIRS radiances on the scales on which they are operationally used, exceeds a threshold so that the additional observations are harmful.
However, the situation changes drastically when observing active, rapidly evolving features with tight gradients. The chosen example is Hurricane Gonzalo. Gonzalo was a TC that formed and developed over the western Atlantic between 12 and 19 October 2014 (Brown 2015). It formed east of the Leeward Islands and started recurving northwestward while undergoing rapid intensification. It reached its peak intensity on 16 October, becoming a category 4 storm with a center pressure of 940 hPa. After landfall over Bermuda, the hurricane accelerated northward and then very rapidly northeastward, while still retaining hurricane strength and a tropical structure as far north as 45°N. Extratropical transition (ET) began at about 50°N with Gonzalo’s remnants becoming a very intense midlatitude cyclone that produced, according to the Met Office, very strong winds on the British Isles (https://blog.metoffice.gov.uk/2014/10/21/top-uk-wind-speeds-as-gonzalos-remnants-felt/). Gonzalo is a good case for this research because its track was intercepted by several passes of the Aqua satellite, thus guaranteeing almost optimal AIRS coverage, which facilitates the investigation of the impacts due to changes in AIRS data and thinning strategy.
Figure 4 shows the zonal vertical cross section and horizontal transects at 850 hPa, across the center of Hurricane Gonzalo, in the RAD, RAD3, and SThin analyses, at 1200 UTC 16 October 2014, the time of its maximum intensity. The RAD cross section shows a reasonable representation of the storm, but it is affected by an excessive asymmetry not supported by observations and is much weaker than observed, since winds of above 60 m s−1 at 700 hPa were reported (Brown 2015). The RAD3 analysis shows the impact of reducing the AIRS data density over the storm: while the RAD3 global skill increased substantially, the relative absence of AIRS information over the storm (recall Figs. 1 and 2) leads to a deterioration of the TC structure compared to RAD. Intensity is drastically reduced, and the asymmetry is further increased. However, the TC structure resulting from the SThin adaptive thinning is substantially better, with a tighter eye, a stronger warm core, and a more symmetric structure. Since the SThin global skill is virtually indistinguishable from RAD3 (Fig. 2), and the TC structure appears to have been improved, the adaptive strategy of combining the global aggressive AIRS thinning of RAD3 with the denser data coverage around Gonzalo appears to be a good compromise.
(top) Zonal vertical section of wind (m s−1), temperature (°C; solid black contours), and the temperature anomaly (solid red contours; contours every 2°C, only ≥4°C for clarity) for Hurricane Gonzalo at 1200 UTC 16 Oct 2014, comparing the RAD, RAD3, and SThin experiments. (bottom) Horizontal winds at 850 hPa (m s−1; shaded) and SLP (hPa; solid).
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
These findings are consistent with those for other TCs present during the experiment period (not shown) and can be interpreted, following LR03, as evidence of a scale-dependent error correlation affecting the assimilation of AIRS radiances.
In fact, these OSEs are an indication that infrared data from hyperspectral instruments could be thinned much more aggressively, because error correlation could be present on a global scale at the current thinning level (as used in OPS or RAD), and that the consequent volume of AIRS radiances assimilated globally is excessive. RAD3 consistently performs better than RAD, hence the intuitive response to simply increase the size of the thinning box to assimilate a lower density of radiances. However, since the analyzed structure of TCs appears to benefit from additional AIRS data on the TC scale, it is likely that error correlation is much lower over the spatial and temporal scales affected by TCs. RAD2, in spite of having a much lower skill globally, improves the TC analysis compared to RAD (not shown), while RAD3 acts in the opposite way: it increases the global skill but degrades the analyzed TC structure. The SThin experiment, which combines RAD2 and RAD3 densities, suggests a scale dependence for error correlation, in agreement with the conceptual understanding offered by LR03, and indicates a TC-centered adaptive thinning as a possible pathway.
The reason why SThin global skill is close, but not superior, to RAD3, may be also interpreted in light of the understanding provided by LR03, concerning an error correlation threshold beyond which additional observations are harmful. In the experiments herein described, the threshold is not known: the fixed size of the TC domain, which does not take into account TC scale or the particular stage in the TC life cycle, may include denser data coverage at times and locations in which additional data are harmful. While the SThin configuration was selected as the best among those examined (recall Table 1), it is undoubtedly suboptimal, because it cannot rigorously separate observations that are beneficial from the ones that are not. A more refined approach could involve variable TC domain sizes, based on the TC scale (which is available from the TC Vitals information).
5. Use of the adjoint of the forecast model
The evidence provided so far by the first set of OSEs is indicative that with the current thinning strategy for AIRS clear-sky radiances, correlated observation errors may be negatively impacting the large scales. However, these correlations do not appear to have a negative impact on the scale of tropical cyclones. This suggests that a simple strategy, focused on assimilating fewer AIRS radiances globally, except around TCs, could be successful.
Additional evidence can be provided with the use of a different methodology to assess observational impacts: the adjoint-based forecast sensitivity observation impact (FSOI) technique, first proposed by Langland and Baker (2004). Benefits arising out of OSEs and adjoint models have been studied and extensively compared, for example within the context of The Observing System Research and Predictability Experiment (THORPEX; created in 2003 by the World Meteorological Organization as a test bed to compare advances and theoretical understanding in data assimilation and observing strategies), by Rabier et al. (2008). Specifically, it is clarified that OSEs and techniques based on the adjoint of the forecast model are complementary methodologies to assess contributions of different observing systems. Gelaro and Zhu (2009) discuss in detail the usefulness and potential of OSEs and adjoint models. Gelaro et al. (2010) compare the impact of assimilated observations on short-range forecast errors in three different forecast systems using an adjoint-based FSOI.
The construction of the GMAO adjoint-based FSOI system follows Langland and Baker (2004) and Trémolet (2008). Two forecasts are integrated 24 h beyond the analysis time of interest: one initialized from the analysis state itself and one from the background state. The reduction in forecast error from the background to the analysis forecast results only from the observations assimilated at analysis time. A response function provides a scalar measure of the forecast error by applying a moist energy norm. The gradient of this function with respect to the model variables is integrated backward to the analysis time 24 h prior using the adjoint of the model. The output from the adjoint describes the regions that are most sensitive to the growth of the forecast errors (in a tangent linear sense) at the initial time. In effect, changes in these locations would have a larger impact on the size of the forecast errors than changes elsewhere, and observations in these locations have the largest impact. Translation of this model sensitivity onto each observation is achieved with an integration of the adjoint of the data assimilation system (Trémolet 2008; Gelaro et al. 2010).
The combined use of adjoints and OSEs represents a powerful tool, because the adjoint-based FSOI can be used to analyze the effects on all the observations of different thinning box sizes applied to AIRS. The adjoint of GEOS benefits from a full suite of moist-physics schemes (Holdaway et al. 2014, 2015), allowing for proper interpretation of observation impact where moist processes are important.
Two FSOI configurations, global and Gonzalo centric, are considered in order to assess the overall impact of AIRS observations. In the global configuration, the adjoint is initialized for the entire global domain, and the impacts are also measured globally. In the Gonzalo-centric configuration, the adjoint is initialized for a limited domain that encompasses the entire life cycle and track of Hurricane Gonzalo. For this configuration, the impact is assessed only within another domain around the initialization domain. This is done to prevent including spurious impacts from remote observations, which can occur because of sensitivity to gravity wave structure and through observation weighting, and is important for constructing informative normalized metrics. Globally, the FSOIs are computed at 0000 UTC. For the Gonzalo-centric domain, the FSOIs are computed at 0600 and 1800 UTC, consistent with the timing of AIRS overpasses. Figure 5 shows the track and intensity of Hurricane Gonzalo with the two domains encompassing the storm. The inner domain (spanning from 75° to 35°W and from 10° to 40°N) shows the response function domain, the outer domain (spanning from 115W° to 5°E and from 30°S to 85°N) is the region in which the impact is assessed. The outer domain is chosen so that 95% of the total observation impact is retained.
Domains for measuring the impact of AIRS observations close to Gonzalo using the adjoint-based FSOI. The inner domain (black lines) shows where the response function for the adjoint sensitivity is computed, i.e., the region where the impact is measured. The outer box (red lines) shows the region for which impacts are screened. AIRS observations occurring outside this region are not included in subsequent metrics. Gonzalo’s track from NHC is superimposed, with colors corresponding to intensity (green, tropical depression; yellow, tropical storm; red, hurricane; purple, major hurricane); extratropical phase not plotted.
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
Figure 6 shows the fraction of beneficial observations resulting from the RAD, RAD2, and RAD3 experiments, computed using the global FSOI configuration. Here, the term “beneficial” refers to observations that reduce the 24-h global error, as measured by the response function. The fraction of beneficial AIRS observations slightly decreases in RAD2 and increases in RAD3. This, together with the changes in global skill seen in Fig. 2, suggests that error correlation is reduced by decreasing the AIRS data density, in agreement with LR03. However, the fact that CrIS data also respond in the same way to the increase and decrease of AIRS data density, even if the CrIS data density is not modified in these experiments, is remarkable and strongly suggestive that cross-instrument error interactions should be considered. This is reasonable because of the similar orbits, but is beyond the scope of this article and will be addressed by a study in preparation. However, for the purposes of this article, it is most important to verify if the fraction of beneficial observations changes when affected by rapidly evolving meteorological features.
Fraction of observations from each instrument type that reduced the 24-h forecast error during the period 1–31 Oct 2014, global domain. Red, green, and blue bars show the impacts on the fraction of beneficial observations produced by the RAD, RAD2, and the RAD3 experiments, respectively.
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
To this purpose, the same impacts seen in Fig. 6 are recomputed but with the response function calculated over a smaller spatial and temporal domain. Figure 7 shows the fraction of beneficial observations between 0600 UTC 12 October and 1800 UTC 19 October, computed using the Gonzalo-centric FSOI configuration. The response function that initializes the adjoint spans from 75° to 35°W and from 10° to 40°N; fractions are computed for a domain spanning from 115°W to 5°E and from 30°S to 85°N. The figure shows the average across all 0600 and 1800 UTC analyses, the approximate time Aqua passes over the domain, from 12 to 19 October 2014. The most striking result is the increase of the AIRS radiances impact in the RAD2 case, indicating that a much denser data distribution is beneficial over the spatial and temporal scales that are affected by Hurricane Gonzalo. It is worth noting that the adjoint forecasts are computed twice, at the times in which the area over which Gonzalo is active is affected by direct AIRS passes, which increases our confidence in the results. Moreover, the environment that controls Gonzalo’s large-scale low-level moist flow largely exceeds the scale of the domain in which the observation impacts are evaluated. Specifically, the easterly flow entering the domain from the east and associated with the African easterly jet is a very sensitive feature characterized by strong meridional temperature gradients, where the detailed infrared information provided by additional AIRS data can provide a beneficial impact on the analysis, as shown in previous studies (e.g., Reale et al. 2009a).
Fraction of observations from each instrument type that reduced 24-h forecast error during the period 12–19 Oct 2014, over a domain encompassing the track and life cycle of Hurricane Gonzalo. Adjoint initialized between 75° and 35°W and between 10° and 40°N. Impact measured between 115°W and 5°E and from 30°S to 85°N. Red, green, and blue bars show the impacts on the fraction of beneficial observations produced by the RAD, RAD2, and RAD3 experiments, respectively. The impacts are computed as the average of the adjoint initialized across all 0600 and 1800 UTC analyses, corresponding approximately to the Aqua passes over the domain.
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
It is important to note the difference between Figs. 6 and 7 in combination with the results discussed concerning Figs. 2 and 4: denser coverage in RAD2 decreases forecast skill and AIRS impacts in global metrics, but it can increase forecast skill and AIRS impacts for Gonzalo. Conversely, the sparser data coverage of RAD3 results in an increase in skill and the AIRS impact globally but a decrease for Gonzalo. These results in unison are strongly suggestive that error correlation acts on a smaller scale in areas affected by hurricanes and that a TC-centered adaptive approach may be beneficial.
6. Cloud-cleared experiments setting
Table 2 and Fig. 8 describe the configuration of the experiments assimilating cloud-cleared radiances and illustrate their corresponding data density. The experiment named CLD differs from RAD only in that cloud-cleared radiances are assimilated instead of those of clear sky. The global data density, consequent to a thinning box choice of 145 km, is about the same. However, the specific distribution of the data is not the same, as can be noted by comparing Figs. 8 and 1. Clear-sky and cloud-cleared radiances are completely different products and as such cannot be expected to correspond precisely. As discussed by Reale et al. (2012), which focused on retrievals, the underlying cloud-clearing algorithm uses a different channel selection than clear sky, and the cloud-cleared radiances are extracted from an array of 3 × 3 FOVs, so an exact correspondence of the data location cannot be expected even if the thinning box is the same. Aside from the precise location of individual assimilated observations, it should be noted that the mere position of CLD observation minus forecast (O-F) values reveals slightly more meteorological structure around the center of Gonzalo than RAD does. This stems from the ability of the cloud-clearing algorithm to extract data from areas that are affected by partial cloud coverage. As for the clear-sky case, two extreme “bounding” experiments, named CLD2 and CLD3, are performed. The CLD2 data distribution seen in Fig. 8 results from a thinning box of 75 km that produces about 4 times more data than CLD, whereas the CLD3 data distribution is obtained by thinning cloud-cleared radiances with a thinning box of 300 km, which retains about one-quarter of the CLD data. Two adaptive thinning selections of cloud-cleared data are discussed in this work: SThin_CLD, combining the data density of CLD3 globally and the data density of CLD2 inside a moving domain of 15° × 15° centered around any TC in the globe, and activated by the BTD information, and SThin2_CLD, which combines CLD density inside the TC domain and CLD3 globally.
As in Table 1, but for the cloud-cleared radiances experiments setup. In the fifth and sixth columns, the comparison is to the CLD experiment.
Cloud-cleared AIRS radiance coverage in all the experiments at 0600 UTC 15 Oct 2014. The boldface L is Hurricane Gonzalo’s center, from the NHC. Circles indicate locations of assimilated radiances: O-F brightness temperatures (K), channel 169, corresponding to approximately 185 hPa.
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
7. Cloud-cleared experiments results
a. Impact on global skill
Figure 9 shows the anomaly correlation of all the cloud-cleared experiments compared to OPS and RAD, which are both used as the reference. As for the clear-sky experiments, the bounding experiments CLD2 and CLD3 produce the highest and lowest skill, respectively, but the drop in skill seen in the CLD2 case is much more dramatic than in RAD2 (recall Fig. 2), indicating that an excessively dense coverage in cloudy radiances is even more harmful. However, the two adaptive thinning experiments, both with a global data density of about one-quarter that of CLD, are clustered together without significant difference in skill from RAD. Moreover, none of them is worse than OPS, which was the operational GEOS version at the time, identical to RAD except for the use of the vortex relocator. The temperature RMSE as a function of height over the tropics, computed for CLD, CLD2, and CLD3 against RAD, for the 24- and 120-h forecasts shows a consistent pattern (and a behavior similar to that noted in Fig. 3 for the bounding clear-sky experiments): CLD3 very slightly outperforms RAD while CLD2 causes a large increase in error (not shown).
Global anomaly correlation for AIRS cloud-cleared radiance assimilation experiments, for the forecasts initialized from 21 Sep to 31 Oct 2014 (41 7-day forecasts, 500-hPa height). (top) Forecast skill as a function of forecast time, (middle) experiment minus OPS forecast skill difference, and (bottom) experiment minus RAD forecast skill difference. The horizontal bars represent statistical significance.
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
This suggests that errors of cloud-cleared radiances are much more correlated than those of clear-sky radiances. Following Daley (1996), the observation error can be separated into an instrumental error and a representativeness error. It is possible that the representativeness error of the cloud-cleared radiances is smaller and as such, the errors are more correlated if assimilated at the same density as the clear-sky radiances. Whatever the cause, the important fact is that cloud-cleared radiances, if assimilated with a data density of about one-quarter that of the currently assimilated clear-sky radiances, can produce a comparable, or even slightly higher, level of global skill in the GEOS.
b. Impact on tropical cyclone analysis
Since the purpose of this work is to improve the analyzed representation of tropical cyclones by making better use of AIRS, three TCs that occur during the period are investigated: Edouard and Gonzalo in the Atlantic and Vance in the eastern Pacific. Of these, Gonzalo is the most favorable from an AIRS coverage perspective. However, cloud-cleared radiances provide a positive impact on all three TCs.
Edouard can be considered as a slowly developing Cape Verde system since its origins can be traced to an African easterly wave that exited the coast of western Africa on 6 September, with closed circulation sufficiently defined to be a tropical depression only five days later, at 1200 UTC 11 September 2014 (Stewart 2014) at about 35°W. Edouard underwent rapid intensification, becoming a category 3 hurricane and reaching peak intensity on 16 and 17 September, and then rapidly weakening on 18 September, while turning eastward, embedded in the westerly flow. From the point of view of this research, it represents a less than optimal case, because the storm quickly deepens and quickly dissipates, not benefiting from the information that several Aqua passes over the mature storm would have provided. In spite of the coverage limitation, the use of AIRS cloud-cleared radiances positively affects the representation of the storm.
In fact, Fig. 10 illustrates the analyzed representation of center pressure for Edouard in three experiments (RAD, CLD3, and SThin2_CLD), compared to the observed best-track data. Results from SThin_CLD are not shown for clarity, being consistently between RAD and SThin2_CLD, as well as for other cases. In other words, SThin2_CLD provides the best representation of TCs throughout the experiment period on all basins. Both analyses that assimilate cloud-cleared data produce a representation of the storm closer to the observations, with slightly deeper values than RAD. However, the zonal vertical cross section of Edouard taken at the time of its second intensity peak (1800 UTC 17 September) shows more clearly (Fig. 11) that the horizontal and vertical structure of the storm is positively affected by the assimilation of cloud-cleared data and that the adaptive thinning methodology brings the largest intensification. Note for example the circulation becoming closed in the northwest quadrant in the SThin2_CLD experiment, and the corresponding increase in vertical extent of the thermal anomaly. While the actual strength of Edouard is still underestimated, the SThin2_CLD experiment shows an intensification of about 10 m s−1 compared to RAD.
Analysis of minimum center pressure for Hurricane Edouard in the RAD, CLD3, and SThin2_CLD experiments, compared to the observations (NHC BTD).
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
(top) Zonal vertical section of wind (m s−1), temperature (°C; solid black contours), and the temperature anomaly (solid red contours; contours every 2°C, only ≥4°C for clarity) for Hurricane Edouard at 1800 UTC 17 Sep 2014, comparing the RAD, CLD3, and SThin2-CLD experiment results. (bottom) Horizontal winds at 850 hPa (m s−1; shaded) and SLP (hPa; solid).
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
Gonzalo’s analyzed center pressure is shown in Fig. 12. The improvement brought by adaptively thinned cloud-cleared radiances is much more dramatic, reaching maximum intensity at 1800 UTC 16 October, just 6 h after peak intensity (when the observed center pressure was still very low, at 942 hPa), and 20 hPa deeper compared to RAD. Gonzalo represents an almost ideal situation for the adaptive thinning strategy; because it is a sufficiently long-lived storm (and hence is covered by multiple Aqua passes), it does not undergo dramatic scale changes, and it is neither too small (which would pose resolution problems) or too large (with a circulation exceeding the swath of the AIRS passes).
As in Fig. 10, but for Hurricane Gonzalo.
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
The importance of good AIRS coverage is particularly evident in Fig. 13, which shows the zonal vertical cross sections across Gonzalo, and the corresponding horizontal sections at 850 hPa, in the RAD, CLD3 and SThin2_CLD cases at 1800 UTC 16 October 2014, very close to its peak intensity. The assimilation of cloud-cleared radiances at a CLD3 density level does not exert an impact on Gonzalo’s structure compared to RAD, but the denser coverage of SThin2_CLD provides a dramatic speed increase of more than 15 m s−1, with peak speed of about 45 m s−1 at about 700 hPa, which is the level where the maximum speed of 68 m s−1 was measured by an U.S. Air Force Hurricane Hunter aircraft at 0000 UTC 17 October (Brown 2015).
As in Fig. 11, but for Hurricane Gonzalo at 1800 UTC 16 Oct 2014.
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
Vance was a small-scale, relatively short-lived hurricane, which lingered between 30 October and 2 November at a tropical depression intensity level, underwent a rapid intensification on 2 November, reached category 2 and peak intensity on 3 November, and very rapidly dissipated on 4 November. This represents a very difficult case for a global DAS, because of the intrinsic problems associated with poor data coverage and resolution. Figure 14 shows the analyzed minimum center pressure as a function of time and confirms that the storm was almost not seen by the DAS in the RAD case. The center pressure never reaches 1000 hPa and does not show any hint of intensification. The cloud-cleared cases are slightly better, both going below 1000 hPa, although strongly underestimating its depth. However, the impact of cloud-cleared radiances on the storm structure is not negligible (Fig. 15). While the RAD vertical section and circulation reveal an open-wind structure, with a shallow warm core not reaching the midtroposphere, the CLD3 case shows a better-defined velocity vertical structure, a more pronounced warm core reaching 300 hPa, and a well-defined low-level circulation. Interestingly, the SThin2_CLD experiment, while better than RAD, does not outperform CLD3. The AIRS coverage (not shown) is relatively poor for this short-lived storm: there are no Aqua passes over Vance at its peak intensity, suggesting that CLD3, which probably produces a slightly better representation of the global scale (recall the better CLD3 global skill in Fig. 9), influences the representation of Vance more than SThin2_CLD because no additional AIRS data are assimilated in spite of the adaptive strategy.
As in Fig. 10, but for Hurricane Vance.
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
As in Fig. 11, but for Hurricane Vance at 1200 UTC 3 Nov 2014. Red contours are for ≥2°C.
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
c. Impact on tropical cyclone track and intensity forecast
The assimilation of adaptively thinned cloud-cleared AIRS radiances does not have a significant impact on error track statistics during the chosen period, as for the case of clear-sky radiances. Compared to RAD, track errors produced by most configurations differ little. For example, the SThin_CLD track error at day 3 for Gonzalo is about 20 km smaller than RAD, whereas SThin2_CLD produces a 72-h track error 30 km larger than RAD. As a reference, the RAD error was about 290 km, slightly larger than the corresponding errors of 250 km by the NCEP GFS and 265 km of the ECMWF model (Brown 2015). The impact of the adaptive strategies on forecast track error for the other storms investigated in these experiments is also negligible (not shown).
However, the impact is significant in the intensity forecast, up to 48 h. This is reasonable because of the improved analyzed TC structure. In particular, the type of impact appears to be different for storms that have a good degree of AIRS coverage during their life cycle and storm that do not. Figure 16 showcases two representative cases, Gonzalo and Vance, showing the RMS error for the predicted center pressures of the two storms as a function of forecasting time. For Gonzalo, there is a significant improvement in the SThin2_CLD case, which is consistent with the substantially improved analyses seen in Figs. 12 and 13. The impact on intensity forecast in the CLD3 is, on the contrary, negligible.
RMS error for center pressure forecast (hPa) as a function of forecast time: (top) Gonzalo and (bottom) Vance.
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
In the case of Vance, the situation is different, with CLD3 giving the best forecast and SThin2_CLD providing a smaller improvement compared to RAD. This is consistent with the fact that Vance is small scale, not well resolved at the GEOS resolution, and that the time of full intensity is short and not covered by a full AIRS pass. As such, the additional information provided by the adaptive thinning is not used. However, the information provided by the cloud-cleared radiances on the large scale brings some improvement in both cases, consistent with the slightly improved analyses seen in Figs. 14 and 15. It is interesting to observe that the impact on intensity forecast decreases as a function of forecasting time in Gonzalo’s case and increases in Vance’s case. This can be explained by considering that the improvement is brought by the additional information on the TC structure provided by adaptive thinning in Gonzalo’s case, and by the improvement brought on the large scale, which is likely the beneficial factor in Vance’s case.
This contrasting behavior, while confirming that cloud-cleared radiances are beneficial in both cases, suggests that a more refined adaptive strategy should include a scale dependence for the size of the TC domain in which denser coverage is to be used instead of a fixed TC domain, as was done in these experiments. Moreover, the limitations caused by insufficient AIRS coverage for short-lived storms could be a consequence of the 3DVar assimilation and may be mitigated by a four-dimensional assimilation. However, it is important to clarify that the cloud-cleared adaptive approach has never produced a negative impact on any of the TCs present during this period over all basins: the impact is strongly positive when coverage is good or neutral when coverage is not optimal (not shown).
Finally, the adjoints of the forecast and analysis models are used in the same way as in the clear-sky case, with focus on the cloud-cleared experiments CLD3 and CLD, and the adaptive thinning SThin2_CLD. The same two FSOI configurations, global and Gonzalo centric (recall Fig. 5), are considered in order to assess the overall impact of AIRS cloud-cleared observations. In the global configuration, the adjoint is used to assess 0000 UTC analysis times for the entire global domain, and the impacts are also measured globally. In the Gonzalo-centric configuration, the adjoint is used to assess the 0600 and 1800 UTC analysis times for a limited domain, which encompasses the entire life cycle and track of Hurricane Gonzalo. The results presented below show the average observation impacts for the two analysis times. By assessing 0600 and 1800 UTC analyses, all times that Aqua flies over the domain are taken into consideration. Figure 17 shows the fraction of beneficial observations on the global domain, comparing the experiments CLD, CLD3, and SThin2_CLD; Fig. 18 compares the fraction of beneficial observations over the Gonzalo-centric domain. On the global scale, CLD3 has the largest fraction of beneficial observations, in agreement with the best global skill seen in Fig. 9. However, within the Gonzalo-centric framework, it is the SThin2_CLD configuration that contains the largest fraction of beneficial observations. This suggests that the adaptive thinning of cloud-cleared radiances is most effective, even more so than when applied to clear-sky observations. This is likely due to the much higher information content in the cloud-cleared radiances. It is worth remembering, for example, that one cloud-cleared radiance is obtained from up to nine field of views within the same footprint (Chahine et al. 2006; Susskind et al. 2011).
Fraction of observations from each instrument type that reduced the 24-h forecast error during the period 1–31 Oct 2014, global domain. Red, blue, and yellow bars show the impacts on the fraction of beneficial observations produced by the CLD, CLD3, and the SThin2_CLD experiment, respectively.
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
Fraction of observations from each instrument type that reduced 24-h forecast error for the period 12–19 Oct 2014, over a domain encompassing the track and life cycle of Hurricane Gonzalo. Adjoint initialized between 75° and 35°W and between 10° and 40°N. Impact measured between 115°W and 5°E and from 30°S to 85°N. Red, blue, and yellow bars show the impacts on the fraction of beneficial observations produced by the CLD, CLD3, and SThin2_CLD experiment, respectively (averaging the forecasts initialized at 0600 and 1800 UTC).
Citation: Weather and Forecasting 33, 4; 10.1175/WAF-D-17-0175.1
8. Discussion
Several caveats are necessary to place this work into a proper context. First, the focus of the research is on AIRS. Ongoing work at this time has shown that the error correlation problem does not affect just AIRS, but also other hyperspectral instruments such as CrIS and IASI. Specifically, the evidence suggests not only that the issue of excessive data density at the current thinning levels may exist for other instruments but also that an interaction between data from different instruments is possible. As a consequence, a more robust strategy should consider revisiting the thinning for the three instruments together. While this is beyond the purpose of this work, it is important to state that the thinning levels that appear to be the best for AIRS from this work could need further adjustments when changes in thinning for other sensors are also included.
Second, the results obtained from the adaptive thinning reveal a scale dependence and are strongly sensitive to coverage. In fact, the specific adaptive thinning schemes seem to benefit different TCs in different domains. For example, Gonzalo is a very favorable case because several passes occur over its lifetime, and its scale does not exceed the swath of the AIRS pass. For Vance, its rapid evolution and small scale make the adaptive configurations used here of little use. For other storms not shown in this work, such as Typhoon Vonfong, whose effects on the large-scale environment largely exceed the swath of AIRS, the response is essentially neutral to AIRS thinning changes.
Third, the limitations caused by the 3DVar approach could act in two directions. From one side, it is possible that the limitations caused by coverage are reduced once observations are continuously assimilated in time. From the other, it is possible that the thinning levels suitable for a 3DVar configuration need to be altered in a 4DVar system.
Fourth, cloud-cleared radiances have not been studied extensively within a data assimilation context. A deeper investigation of their quality control could be beneficial to understanding their different responses to thinning. It is possible that the representativeness error of cloud-cleared radiances could be smaller than clear-sky radiances, since they represent both clear areas and areas that are partly affected by clouds, and result from an average of more FOVs, which reduces their signal-to-noise ratio. A smaller representativeness error could be one cause of larger error correlation. While this lies beyond the scope of this article, it is nevertheless an important aspect to consider in future investigations aiming at the operational use of cloud-cleared radiances.
Fifth, only the horizontal error correlation problem is addressed in this work. The problem of correlation between channels (e.g., Todling et al. 2015) is not investigated. It is possible that cross-channel error correlation could be altered by the cloud-clearing procedure.
Sixth, while the impact of cloud-cleared radiances from AIRS and CrIS has been proven to be beneficial within a regional model context with a focus on TCs (Wang et al. 2015, 2017), and the possibility of adaptive thinning could be explored, the results from this work would probably not be directly applicable without additional research. The concept of “optimal data density” could be drastically different within a regional model context. For example, Lin et al. (2017) illustrates the beneficial impact of AIRS on short-range forecasts even in a data-rich area, after undergoing a preprocessing that involves channel selection and bias-correction spinup.
9. Conclusions
Notwithstanding the limitations discussed in the previous section, there are two findings suggested by the two sets of OSEs and the adjoint results herein described. The first appears to be evidence that AIRS clear-sky radiance errors at the current thinning level are strongly correlated on a global scale but that error correlation is reduced around tropical cyclones. This finding is intuitively consistent with the conceptual understanding provided by LR03.
The second finding is that there is great potential benefit in using cloud-cleared radiances, rather than clear-sky radiances, within an operational context. Their primary effect on TC representation (consistent with already known impacts of assimilating cloud-cleared retrievals; e.g., Reale et al. 2009b) is to create a very strong and concentrated temperature anomaly in the upper troposphere (e.g., Fig. 13), with gradients on the order of more than 10°C (100 km)−1, which translates, through hydrostatic adjustment, to a lower central pressure. The thermal anomaly is present for all TCs worldwide (not shown) and has the combined effect of deepening the storms and also adjusting their positions. In fact, the peak of the temperature anomaly that is induced by the assimilation of cloud-cleared radiances is aligned with the storm center. Consequently, the use of cloud-cleared radiances negates the need for a vortex relocator, which has been noted in this paper to reduce global skill (recall the comparison of OPS with RAD in Figs. 2 and 9) and has a computational cost. Cloud-cleared radiances are distributed from the Goddard DAAC and are available online in real time (https://daac.gsfc.nasa.gov/datasets/AIRI2CCF_V006/summary?keywords=AIRS%20cloud%20cleared%20radiances). Alternatively, operational centers could produce their own cloud-cleared radiances by developing an algorithm or modifying existing ones (e.g., Susskind et al. 2014), which could improve latency. The errors of cloud-cleared radiances appear to have higher spatial error correlations than clear-sky radiances, meaning they can and should be thinned more aggressively. A density of about one-quarter of the cloud-cleared radiances results in retaining the same global skill as RAD, while substantially improving the TC analysis. These facts should be considered in an operational environment, because they indicate the possibility for reduced computational costs.
The underlying motivation for this article was to explore the possibility of error correlation affecting the forecast skill in response to the assimilation of AIRS radiances. OSEs with different adaptive configurations suggest that the scales of observational error correlation for AIRS are much smaller for TCs. The use of the adjoint of the forecast model is supportive of this idea and encourages further research on a simple, TC-based, adaptive scheme for all hyperspectral data. In addition, this work shows that cloud-cleared radiances, if used at a much sparser level than clear-sky radiances globally except around tropical cyclones, can bring a comparable or even slightly superior level of global skill, while drastically improving the TC analysis and intensity forecasts in the GEOS. The implications for TC operational forecast improvement are noteworthy and worth exploring.
Acknowledgments
The authors gratefully acknowledge support by Dr. Ramesh K. Kakar (NASA HQ) through NASA Grant NNX14AK19G and Dr. Tsengdar Lee (NASA HQ) for allocations on NASA High-End Computing resources. All simulations were performed at the NASA Center for Climate Studies (NCCS) in Greenbelt, Maryland. AIRS data are distributed by the NASA Distributed Active Archive Center (DAAC) Goddard Earth Sciences Data and Information Services Center (GES DISC).
REFERENCES
Bauer, P., A. J. Geer, P. Lopez, and D. Salmond, 2010: Direct 4D-Var assimilation of all-sky radiances. Part I: Implementation. Quart. J. Roy. Meteor. Soc., 136, 1868–1885, https://doi.org/10.1002/qj.659.
Bishop, C. H., and Z. Toth, 1999: Ensemble transformation and adaptive observations. J. Atmos. Sci., 56, 1748–1765, https://doi.org/10.1175/1520-0469(1999)056<1748:ETAAO>2.0.CO;2.
Bondarenko, V., T. Ochotta, and D. Sauple, 2007: The interaction between model resolution, observation resolution and observation density in data assimilation: A two-dimensional study. 11th Symp. on Integrated Observing and Assimilation Systems for the Atmosphere, Oceans, and Land Surface, San Antonio, TX, Amer. Meteor. Soc., 5.19, https://ams.confex.com/ams/87ANNUAL/techprogram/paper_117655.htm.
Brown, D. P., 2015: Hurricane Gonzalo (AL082014), 12–19 October 2014. National Hurricane Center Tropical Cyclone Rep., 30 pp., www.nhc.noaa.gov/data/tcr/AL082014_Gonzalo.pdf.
Chahine, M. T., and Coauthors, 2006: AIRS: Improving weather forecasting and providing new data on greenhouse gases. Bull. Amer. Meteor. Soc., 87, 911–926, https://doi.org/10.1175/BAMS-87-7-911.
Daley, R., 1996: Atmospheric Data Analysis. Cambridge University Press, 457 pp.
Derber, J., and W.-S. Wu, 1998: The use of TOVS cloud-cleared radiances in the NCEP SSI analysis system. Mon. Wea. Rev., 126, 2287–2299, https://doi.org/10.1175/1520-0493(1998)126<2287:TUOTCC>2.0.CO;2.
Gelaro, R., and Y. Zhu, 2009: Examination of observation impacts derived from observing system experiments (OSEs) and adjoint models. Tellus, 61A, 179–193, https://doi.org/10.1111/j.1600-0870.2008.00388.x.
Gelaro, R., R. H. Langland, G. D. Rohaly, and T. E. Rosmond, 1999: An assessment of the singular vector approach to targeted observing using the FASTEX dataset. Quart. J. Roy. Meteor. Soc., 125, 3299–3328, https://doi.org/10.1002/qj.49712556109.
Gelaro, R., R. H. Langland, S. Pellerin, and R. Todling, 2010: The THORPEX Observation Impact Intercomparison Experiment. Mon. Wea. Rev., 138, 4009–4025, https://doi.org/10.1175/2010MWR3393.1.
Gelaro, R., and Coauthors, 2017: The Modern-Era Retrospective Analysis for Research and Applications, version-2 (MERRA-2). J. Climate, 30, 5419–5454, https://doi.org/10.1175/JCLI-D-16-0758.1.
Holdaway, D., R. Errico, R. Gelaro, and J. G. Kim, 2014: Inclusion of linearized moist physics in NASA’s Goddard Earth Observing System data assimilation tools. Mon. Wea. Rev., 142, 414–433, https://doi.org/10.1175/MWR-D-13-00193.1.
Holdaway, D., R. Errico, R. Gelaro, J. G. Kim, and R. Mahajan, 2015: A linearized prognostic cloud scheme in NASA’s Goddard Earth Observing System data assimilation tools. Mon. Wea. Rev., 143, 4198–4219, https://doi.org/10.1175/MWR-D-15-0037.1.
Joo, S., J. Eyre, and R. Marriott, 2013: The impact of MetOp and other satellite data within the Met Office Global NWP system using and adjoint-based sensitivity method. Mon. Wea. Rev., 141, 3331–3342, https://doi.org/10.1175/MWR-D-12-00232.1.
Langland, R. H., and N. L. Baker, 2004: Estimation of observation impact using the NRL Atmospheric Variational Data Assimilation Adjoint System. Tellus, 56A, 189–201, https://doi.org/10.3402/tellusa.v56i3.14413.
Lazarus, S. M., M. E. Splitt, M. D. Lueken, and B. T. Zavodsky, 2010: Evaluation of data reduction algorithms for real-time analysis. Wea. Forecasting, 25, 837–851, https://doi.org/10.1175/2010WAF2222296.1.
Le Marshall, J., and Coauthors, 2006: Improving global analysis and forecasting with AIRS. Bull. Amer. Meteor. Soc., 87, 891–894, https://doi.org/10.1175/BAMS-87-7-891.
Lin, H., S. S. Weygandt, A. H. Lim, M. Hu, J. M. Brown, and S. G. Benjamin, 2017: Radiance preprocessing for assimilation in the hourly updating Rapid Refresh mesoscale model: A study using AIRS data. Wea. Forecasting, 32, 1781–1800, https://doi.org/10.1175/WAF-D-17-0028.1.
Lin, S.-J., 2004: A vertically Lagrangian finite-volume dynamical core for global models. Mon. Wea. Rev., 132, 2293–2307, https://doi.org/10.1175/1520-0493(2004)132<2293:AVLFDC>2.0.CO;2.
Liu, Z.-Q., and F. Rabier, 2002: The interaction between model resolution, observation resolution and observation density in data assimilation: A one-dimensional study. Quart. J. Roy. Meteor. Soc., 128, 1367–1386, https://doi.org/10.1256/003590002320373337.
Liu, Z.-Q., and F. Rabier, 2003: The potential of high-density observations for numerical weather prediction: A study with simulated observations. Quart. J. Roy. Meteor. Soc., 129, 3013–3035, https://doi.org/10.1256/qj.02.170.
Lorenz, E. N., and K. A. Emanuel, 1998: Optimal sites for supplementary weather observations: Simulation with a small model. J. Atmos. Sci., 55, 399–414, https://doi.org/10.1175/1520-0469(1998)055<0399:OSFSWO>2.0.CO;2.
McCarty, W., G. Jedlovec, and T. L. Miller, 2009: Impact of the assimilation of Atmospheric Infrared Sounder radiance measurements on short-term weather forecasts. J. Geophys. Res., 114, D18122, https://doi.org/10.1029/2008JD011626.
McCarty, W., and Coauthors, 2016: MERRA-2 input observations: Summary and assessment. NASA/TM-2016-104606, NASA Tech. Rep. Series on Global Modeling and Data Assimilation, Vol. 46, 61 pp., https://gmao.gsfc.nasa.gov/pubs/.
McNally, A. P., P. D. A. Watts, J. Smith, R. Engelen, G. A. Kelly, J. N. Thépaut, and M. Matricardi, 2006: The assimilation of AIRS radiance data at ECMWF. Quart. J. Roy. Meteor. Soc., 132, 935–957, https://doi.org/10.1256/qj.04.171.
Molod, A., L. Takacs, M. Suarez, J. Bacmeister, I.-S. Song, and A. Eichmann, 2012. The GEOS-5 atmospheric general circulation model: Mean climate and development from MERRA to Fortuna. NASA/TM–2012-104606, NASA Tech. Rep. Series on Global Modeling and Data Assimilation, Vol. 28, 124 pp., https://gmao.gsfc.nasa.gov/GEOS_systems/geos5/index_pubs.php.
Morss, R. E., K. A. Emanuel, and C. Snyder, 2001: Idealized adaptive observation strategies for improving numerical weather prediction. J. Atmos. Sci., 58, 210–232, https://doi.org/10.1175/1520-0469(2001)058<0210:IAOSFI>2.0.CO;2.
Ochotta, T., C. Cebhardt, D. Saupe, and W. Wergen, 2005: Adaptive thinning of atmospheric observations in data assimilation with vector quantization and filtering methods. Quart. J. Roy. Meteor. Soc., 131, 3427–3437, https://doi.org/10.1256/qj.05.94.
Ochotta, T., C. Cebhardt, V. Bondarenko, D. Saupe, and W. Wergen, 2007: On thinning methods for data assimilation of satellite observations. 23rd Int. Conf. on Interactive Information Processing Systems, San Antonio, TX, Amer. Meteor. Soc., 2B.3, http://ams.confex.com/ams/pdfpapers/118511.pdf.
Pangaud, T., N. Fourrie, V. Guidard, M. Dahoui, and F. Rabier, 2009: Assimilation of AIRS radiances affected by mid- to low-level clouds. Mon. Wea. Rev., 137, 4276–4292, https://doi.org/10.1175/2009MWR3020.1.
Purser, R. J., W. Wu, D. F. Parrish, and N. M. Roberts, 2003a: Numerical aspects of the application of recursive filters to variational statistical analysis. Part I: Spatially homogeneous and isotropic Gaussian covariances. Mon. Wea. Rev., 131, 1524–1535, https://doi.org/10.1175//1520-0493(2003)131<1524:NAOTAO>2.0.CO;2.
Purser, R. J., W. Wu, D. F. Parrish, and N. M. Roberts, 2003b: Numerical aspects of the application of recursive filters to variational statistical analysis. Part II: Spatially inhomogeneous and anisotropic Gaussian covariances. Mon. Wea. Rev., 131, 1536–1548, https://doi.org/10.1175//2543.1.
Putman, W. M., and S.-J. Lin, 2007: Finite-volume transport on various cubed-sphere grids. J. Comput. Phys., 227, 55–78, https://doi.org/10.1016/j.jcp.2007.07.022.
Rabier, F., and Coauthors, 2008: An update on THORPEX-related research in data assimilation and observing strategies. Nonlinear Processes Geophys., 15, 81–94, https://doi.org/10.5194/npg-15-81-2008.
Reale, O., J. Susskind, R. Rosenberg, E. Brin, E. Liu, L. P. Riishojgaard, J. Terry, and J. C. Jusem, 2008: Improving forecast skill by assimilation of quality-controlled AIRS temperature retrievals under partially cloudy conditions. Geophys. Res. Lett., 35, L08809, https://doi.org/10.1029/2007GL033002.
Reale, O., W. K. Lau, K.-M. Kim, and E. Brin, 2009a: Atlantic tropical cyclogenetic processes during SOP-3 NAMMA in the GEOS-5 global data assimilation and forecast system. J. Atmos. Sci., 66, 3563–3578, https://doi.org/10.1175/2009JAS3123.1.
Reale, O., W. K. Lau, J. Susskind, E. Brin, E. Liu, L. P. Riishojgaard, M. Fuentes, and R. Rosenberg, 2009b: AIRS impact on the analysis and forecast track of Tropical Cyclone Nargis in a global data assimilation and forecasting system. Geophys. Res. Lett., 36, L06812, https://doi.org/10.1029/2008GL037122.
Reale, O., K. M. Lau, J. Susskind, and R. Rosenberg, 2012: AIRS impact on analysis and forecast of an extreme rainfall event (Indus River valley, Pakistan, 2010) with a global data assimilation and forecast system. J. Geophys. Res., 117, D08103, https://doi.org/10.1029/2011JD017093.
Rienecker, M. M., and Coauthors, 2008: The GEOS-5 Data Assimilation System: Documentation of versions 5.0.1, 5.1.0, and 5.2.0. NASA/TM-2008-104606, NASA Tech. Rep. Series on Global Modeling and Data Assimilation, Vol. 27, 92 pp., https://ntrs.nasa.gov/search.jsp?R=20120011955.
Singh, R., C. M. Kishtawal, and P. K. Pal, 2011: Use of Atmospheric Infrared Sounder clear-sky and cloud-cleared radiances in the Weather Research and Forecasting 3DVAR assimilation system for mesoscale weather predictions over the Indian region. J. Geophys. Res., 116, D22116, https://doi.org/10.1029/2011JD016379.
Stewart, R. S., 2014: Hurricane Edouard (AL062014), 11–19 September 2014. National Hurricane Center Tropical Cyclone Rep., 19 pp., www.nhc.noaa.gov/data/tcr/AL062014_Edouard.pdf.
Susskind, J., C. Barnet, J. Blaisdell, L. Iredell, F. Keita, L. Kouvaris, G. Molnar, and M. Chahine, 2006: Accuracy of geophysical parameters derived from Atmospheric Infrared Sounder/Advanced Microwave Sounding Unit as a function of fractional cloud cover. J. Geophys. Res., 111, D09S17, https://doi.org/10.1029/2005JD006272.
Susskind, J., J. M. Blaisdell, L. Iredell, and F. Keita, 2011: Improved temperature sounding and quality control methodology using AIRS/AMSU data: The AIRS Science Team version 5 retrieval algorithm. IEEE Trans. Geosci. Remote Sens., 49, 883–907, https://doi.org/10.1109/TGRS.2010.2070508.
Susskind, J., J. M. Blaisdell, and L. Iredell, 2014: Improved methodology for surface and atmospheric soundings, error estimates, and quality control procedures: The Atmospheric Infrared Sounder science team version-6 retrieval algorithm. J. Appl. Remote Sens., 8, 084994, https://doi.org/10.1117/1.JRS.8.084994.
Todling, R., W. Gu, and D. N. Daescu, 2015: Accounting for satellite radiance inter-channel correlations in GSI. JCSDA Quarterly, No. 52, Joint Center for Satellite Data Assimilation, College Park, MD, 6–9, https://www.jcsda.noaa.gov/documents/newsletters/2015_04JCSDAQuarterly.pdf.
Trahan, S., and L. Sparling, 2012: An analysis of NCEP tropical cyclone vitals and potential effects on forecasting models. Wea. Forecasting, 27, 744–756, https://doi.org/10.1175/WAF-D-11-00063.1.
Trémolet, Y., 2008: Computation of observation sensitivity and observation impact in incremental variational data assimilation. Tellus, 60A, 964–978, https://doi.org/10.1111/j.1600-0870.2008.00349.x.
Wang, P., and Coauthors, 2015: Assimilation of thermodynamic information from Advanced Infrared Sounders under partially cloudy skies for regional NWP. J. Geophys. Res. Atmos., 120, 5469–5484, https://doi.org/10.1002/2014JD022976.
Wang, P., J. Li, Z. Li, A. H. N. Lim, J. Li, T. J. Schmit, and M. D. Goldberg, 2017: The impact of Cross-track Infrared Sounder (CrIS) cloud-cleared radiances on Hurricane Joaquin (2015) and Matthew (2016) forecasts. J. Geophys. Res. Atmos., 122, 13 201–13 218, https://doi.org/10.1002/2017JD027515.
Wu, W. S., R. J. Purser, and D. F. Parrish, 2002: Three-dimensional variational analysis with spatially inhomogeneous covariances. Mon. Wea. Rev., 130, 2905–2916, https://doi.org/10.1175/1520-0493(2002)130<2905:TDVAWS>2.0.CO;2.
Zhou, Y., W. K. Lau, O. Reale, and R. Rosenberg, 2010: AIRS impact on precipitation analysis and forecast of tropical cyclones in a global data assimilation and forecasting system. Geophys. Res. Lett., 37, L02806, https://doi.org/1029/2009GL041494.
Zhu, T., and S. Boukabara, 2015: Development and impact study of community satellite data thinning and representation optimization tool. 20th Conf. on Satellite Meteorology and Oceanography, Phoenix, AZ, Amer. Meteor. Soc., 225, https://ams.confex.com/ams/95Annual/webprogram/Paper267375.html.
Zhu, Y., E. Liu, R. Mahajan, C. Thomas, D. Groff, P. Van Delst, A. Collard, D. Kleist, R. Treadon, and J. C. Derber, 2016: All-sky microwave radiance assimilation in NCEP’s GSI analysis system. Mon. Wea. Rev., 144, 4709–4735, https://doi.org/10.1175/MWR-D-15-0445.1.