Nonlinear Bias Correction for Satellite Data Assimilation Using Taylor Series Polynomials

1. Introduction

The ability to generate accurate cloud and water vapor (WV) analyses suitable for numerical weather prediction (NWP) models is perhaps the most challenging aspect of modern data assimilation (DA) systems because they typically assume Gaussian error statistics and that linear relationships exist between the observations and model state variables. Cloud processes, however, are inherently nonlinear with complex interactions occurring between different cloud hydrometeor species and the local thermodynamic environment at spatial and temporal scales that are typically much smaller than those represented by NWP models. Likewise, WV content can change rapidly in space and time and can influence the evolution of the cloud field in nonlinear ways. These and other factors can make it very challenging to effectively assimilate information from cloud and WV sensitive observations.

Remotely sensed observations obtained using geostationary and polar-orbiting satellites provide the only reliable source of high-resolution cloud and WV information covering large geographic domains. Sophisticated visible, infrared, and microwave sensors on board various satellite platforms provide information about the spatial distribution and characteristics of the cloud and WV fields. For regional-scale NWP, observations from geostationary satellites are especially useful because their continuous viewing of the same area with high temporal and spatial resolution allow them to more easily constrain the evolution of rapidly changing weather features (Vukicevic et al. 2006; Errico et al. 2007). Satellite observations, however, often exhibit biases when compared to their model equivalents computed using the NWP model background; therefore, bias correction (BC) methods are typically required to assimilate these observations (Eyre 2016).

Observation-minus-background (OMB) biases can occur for a variety of reasons and can differ for clear and cloudy observations. For example, biases can arise from calibration errors in a satellite sensor or to instrument “drift” as a sensor ages. Biases can also be introduced by deficiencies in the forward radiative transfer models used to compute the model equivalent brightness temperatures. For clear-sky observations, biases may result from errors in the specification of surface emissivity, simplifications in the radiative transfer model equations, inadequate vertical resolution or a low model top in the NWP model, or the misspecification or absence of atmospheric constituents (such as aerosols) observed by some satellite bands. In the context of clear-sky DA, biases can also be introduced by incomplete cloud screening procedures that allow some cloud-affected observations to pass quality control and thereby incorrectly enter the DA system. Indeed, most existing quality control methods were originally designed to remove all cloud-affected observations; however, these constraints are being relaxed as operational modeling centers move toward all-sky DA (e.g., Okamoto et al. 2014; Zhu et al. 2016). Exclusion of cloud-affected brightness temperatures has the undesirable consequence of removing observations that could have been used to improve the model initialization in cloudy areas of the model domain.

Additional uncertainties regarding the specification of cloud properties arise when assimilating cloud-affected infrared brightness temperatures. Though forward radiative transfer modeling for cloudy scenes has become more accurate in recent years, deficiencies remain, especially for ice clouds. Simulation of absorption and scattering properties for liquid clouds is relatively straightforward because the droplets are assumed to be spherical. However, there are larger uncertainties with ice cloud bulk optical properties because there is some dependence in the infrared on the shape of the ice particles (Yang et al. 2013). For example, an ice particle may take the form of a hexagonal plate, solid or hollow column, bullet rosette, or an aggregate of some form, and impact the bulk microphysical and optical properties that result from integration of the individual particle properties over the assumed size and habit distributions (Baum et al. 2014). In addition, the ice water path is related to both the cloud optical thickness and the cloud particle effective diameter. When computing simulated brightness temperatures, these diameters should be computed using the particle size distribution and cloud property assumptions made for each cloud species by a given microphysics scheme (e.g., Otkin et al. 2009; Cintineo et al. 2014; Thompson et al. 2016).

Biases in the OMB departures can also be caused by systematic errors in the NWP model forecasts that result from deficiencies in the parameterization schemes or other characteristics of the NWP model. It is well known that model forecasts containing large biases influence the behavior of BC methods and can degrade the performance of DA systems (Dee 2005; Dee and Uppala 2009; Eyre 2016). Biases can be especially large for model variables for which few observations are available to constrain their evolution, such as root zone soil temperature and moisture (Mahfouf 2010), or variables such as clouds and WV that are strongly influenced by parameterization schemes accounting for subgrid-scale processes. For example, uncertainties in microphysical parameters controlling cloud generation and decay processes can lead to systematic errors in the spatial extent, optical thickness, and height of the clouds, which in turn impacts the simulated satellite brightness temperatures (Otkin and Greenwald 2008; Cintineo et al. 2014; Eikenberg et al. 2015). Ideally, a BC method would not remove the bias in the OMB departures associated with deficiencies in the NWP model because the observations should be used to correct such systematic errors. In the absence of a perfect reference analysis, however, it can be very difficult to determine whether a bias originates in the observations or forward radiative transfer model, both of which should be corrected, or in the model background (Dee 2005). Because of this uncertainty in bias attribution, all BC methods functionally act to correct the bias in the “observations” regardless of the true sources of the bias (Dee and Uppala 2009). Though this outcome is not desirable because it will limit the ability of the observations to reduce systematic errors in the analysis, it does satisfy the requirement by most DA methods that the observations are unbiased. In addition, the bias-corrected observations can still be used to reduce random errors in the analysis. Eyre (2016) noted that the impact of model bias on the analysis accuracy depends on the rate at which the NWP model state relaxes back toward its own climatology after the assimilation update. If an NWP model quickly returns to its preferred state, then the analysis errors will continue to be large even if the model bias can be removed prior to computing the BC coefficients. This points toward the need to fix the bias at its source within the NWP model. The impact of model bias on a BC method can be reduced when high-quality “anchor” observations with little or no bias are available; however, it is not apparent that such observations exist for WV and clouds.

BC methods can be broadly categorized into two types (Eyre 2016). The first type uses departures between the observations and their model equivalents accumulated over long time periods outside of the DA system to estimate and remove the bias from the observations prior to their assimilation. These so-called static BC methods typically use the satellite scan angle along with several atmospheric variables, such as the geopotential thickness over some layer, as the BC predictors. The BC coefficients for each satellite sensor and band are then computed using linear least squares regressions between the predictors and the observations. In practice, however, these “static” BC coefficients are regularly updated to account for changes in the model background due to changes in the NWP model or DA system, the addition of new observations, and upgrades to the forward radiative transfer model. Frequent retuning of a static BC method can be beneficial because it makes it more adaptable to changes in the models and observations. More detailed descriptions of static BC methods can be found in Eyre (1992), Harris and Kelly (2001), and Hilton et al. (2009).

With the second type of BC method, known as variational BC (VarBC), the BC coefficients are updated simultaneously with the control vector during each DA cycle using the same set of observations and an augmented control vector (Derber et al. 1991; Parrish and Derber 1992; Derber and Wu 1998; Dee 2005; Auligne et al. 2007; Dee and Uppala 2009; Zhu et al. 2014). Like static BC methods, VarBC typically uses the satellite scan angle and several variables describing the atmospheric state as the predictors, with the total BC treated as a linear combination of all predictors. The BC coefficient for each predictor is computed during the minimization of the variational cost function. With an incremental DA approach with multiple outer loops, the BC coefficient increments evolve during each iteration of the inner loop and are updated at the end of each outer loop, which allows the coefficients to adjust with time and capture changes in observation quality. The state space augmentation approach used by VarBC also requires an estimate of the background covariances of the augmented state vector. For simplicity, most schemes assume that the error for a given BC parameter is uncorrelated with errors in other parameters for other satellite sensors and bands and with errors in the model background (Derber and Wu 1998; Dee 2005).

Most BC methods have been developed for use in variational or hybrid DA systems; however, several studies have also explored BC in ensemble DA systems. Fertig et al. (2009) developed a BC method for ensemble DA that is similar to VarBC in that it uses state augmentation to estimate the biases during the assimilation step. They showed that their method was able to reduce both the observation bias and the analysis error in perfect model experiments. Similar methods have also been used successfully in real data experiments assimilating microwave brightness temperatures (Szunyogh et al. 2008; Aravequia et al. 2011; Miyoshi et al. 2010). In high-resolution observing system simulation experiments assimilating infrared brightness temperatures, Cintineo et al. (2016) found that the analysis and forecast accuracy was improved when a simple fixed-value BC was applied to the clear-sky observations similar to that used by Stengel et al. (2009, 2013) in a variational DA system. Cintineo et al. (2016), however, did not bias correct the cloudy observations prior to their assimilation because their bias was too complex to properly handle using a simple fixed-value BC applied uniformly to all cloudy observations. Zhu et al. (2016) handled biases in all-sky microwave observations by computing the BC coefficients using only cases where both the model background and the observations were either clear or cloudy. By doing this, they were able to reduce errors associated with mismatched cloud fields, while still preserving cloud-dependent information in the matched observations. Together, these results provide evidence that more sophisticated BC methods that can account for changes in cloud properties are necessary to effectively remove biases in the OMB departures.

In this study, we present a new BC method that can be used to diagnose and remove biases in all-sky infrared brightness temperatures using a Taylor series polynomial expansion of the OMB departures. This approach can diagnose both linear and nonlinear bias components through use of higher-order Taylor series terms and a set of BC predictors. For example, with a third-order approximation, the zeroth- and first-order terms represent the constant and linear bias components, whereas the second- (quadratic) and third-order (cubic) terms represent nonlinear bias components. We use this nonlinear BC (NBC) method to remove the bias from Spinning Enhanced Visible and Infrared Imager (SEVIRI) infrared brightness temperatures that were passively monitored during high-resolution ensemble DA experiments. The paper is organized as follows. The DA framework is described in section 2, with a mathematical description of the NBC method presented in section 3. Statistics obtained using the NBC method are shown in section 4, with conclusions and a discussion presented in section 5.

2. Experimental design

3. Nonlinear bias correction (NBC) method

Traditional BC methods remove biases between a given set of observed and model-equivalent satellite brightness temperatures through the use of a set of BC predictors that describe the atmospheric state or characteristics of the satellite data. Both static and VarBC methods typically assume that a linear relationship exists between the observation departure bias and a given set of predictors or that a global constant can be added to the observations. This linear BC approach has been shown to work well for clear-sky observations possessing Gaussian error characteristics for which a set of constant and linear BC coefficients are sufficient to remove the bias; however, their use will be suboptimal if the observation departure bias varies as a nonlinear function of some predictor. For satellite observations, nonlinear error dependencies are more likely to occur when cloudy observations are assimilated given the prevalence of nonlinear processes in clouds that could lead to complex errors in the forecast cloud field and the possibility that nonlinear error sources could be introduced by the forward radiative transfer model used to compute the model-equivalent brightness temperatures. For example, with infrared brightness temperatures, it is possible that increased uncertainty simulating ice radiative properties in forward radiative transfer models could lead to biases that are a nonlinear function of some cloud property, such as cloud-top height. Thus, given the increased interest in all-sky DA, it is desirable to develop BC methods that can remove both linear and nonlinear bias components from the innovations.

One method that can be used to account for nonlinear error dependencies in a set of observations is a Taylor series polynomial expansion that includes higher-order terms that can capture nonlinear features of the error distribution if they exist. For a given set of observed and model-equivalent brightness temperatures corresponding to a specific satellite sensor and band, the observation departure vector is defined as

View Expanded

where y is the observation vector, x is the NWP model state vector, and is the observation operator that is used to compute the model equivalent brightness temperatures. If we assume that the bias in the observation departures can be described by a real function of a single variable (e.g., predictor) that is infinitely differentiable around a real number c, Eq. (1) can be decomposed into an N-order Taylor series expansion:

View Expanded

where dy is the m × 1 observation departure vector and m is the number of observations, is the nth derivative of f evaluated at the point c, and is the predictor value for the ith observation. The notation outside the square brackets indicates that the Taylor series approximation is computed separately for each element of the dy vector using the equation within the brackets. The variable used as the predictor is chosen based on its ability to capture some aspect of the observation departure bias, whereas the value of that variable for a given observation can be obtained from a variety of sources, such as the model background or a satellite retrieval. The constant c can be set to any value because simply moves c to another constant value; therefore, for convenience, we define c to be the mean of the predictor values:

View Expanded

It is readily apparent from Eq. (2) that the higher-order terms represent nonlinear components because the exponents are , with the and polynomials representing the quadratic and cubic terms, respectively.

The single variable case shown in Eq. (2) can subsequently be generalized to be a function of more than one predictor:

View Expanded

which can be written more compactly as

View Expanded

where d is the number of predictors, denotes the nth partial derivative of f evaluated at the point , and is the ith value for a given predictor .

For illustrative purposes, if we assume a single variable, third-order Taylor series expansion for a single satellite sensor and band, and define the BC coefficients such that , Eq. (2) can be written as

View Expanded

or alternatively in matrix notation as

View Expanded

where dy is the m × 1 observation departure vector, is an m × n matrix containing the n Taylor series terms for each ith observation, where , and b is an n × 1 vector containing the BC coefficients. This is an overdetermined system of m linear equations in n unknown coefficients because . The first column of contains ones, with the remaining columns containing the linear and higher-order Taylor series terms. Because this kind of system typically does not have an analytic solution, we instead want to find the coefficients b that best fit the equations by solving the quadratic minimization problem , where the objective function S is given by

View Expanded

and is the Euclidean norm. Because most real-world phenomena act as a low-pass filter in the forward direction where maps to , the inverse mapping will operate as a high-pass filter that amplifies noise and can therefore lead to a poorly conditioned problem. Preference, however, can be given to smaller norms by adding a Tikhonov regularization term, , to Eq. (8), which is a standard approach when solving inverse problems (Nakamura and Potthast 2015). For simplicity, we choose a matrix that is a multiple of the identity matrix , such that

View Expanded

Sensitivity tests showed that α could be set to a very small value () when one variable was used in the regression; however, a slightly larger value () was found to work better for the multivariate regressions. These values were used for the univariate and multivariate experiments presented in section 4. The least squares solution can then be found by differentiating with respect to b, and equating to 0, such that

View Expanded

or alternatively, after rearranging and multiplying both sides of Eq. (10) by , we can solve for the b vector containing the BC coefficients using

View Expanded

where is a symmetric, square matrix with dimensions n × n. The small dimensions of this matrix make it easy to compute its inverse, thereby making it feasible to include higher-order Taylor series terms, additional predictors, and a large OMB departure dataset when computing the BC coefficients. After solving for b, which is done separately for each satellite band and sensor, the BC coefficients can then be applied to dy to remove the linear and nonlinear conditional bias components from the observations.

4. Results

In this section, the ability of the NBC method to remove biases from all-sky satellite infrared brightness temperatures is assessed using OMB departure statistics accumulated at hourly intervals during a 4.5-day period in which conventional observations were actively assimilated and SEVIRI observations were passively monitored. Figure 1 shows the evolution of the observed SEVIRI 6.2-μm brightness temperatures during this time period. At the start of the period on 16 May (Fig. 1a), an area of cold upper-level clouds associated with a band of precipitation was located across the eastern half of the domain. This weather feature slowly weakened over Germany during the next two days (Figs. 1b,c), with the brightness temperatures becoming warmer as the convective clouds were replaced by cirrus and midlevel clouds. Generally clear skies characterized by warm brightness temperatures were also present across parts of the domain during this time period, with clear skies prevailing across most of the region on 19 May (Fig. 1d). A large area of convection with very cold upper-level clouds then moved into the western half of the domain on 20 May (Fig. 1e). Overall, it is evident that the study period contains a wide range of atmospheric conditions and cloud types that supports a realistic assessment of the NBC method during the warm season.

View Full Size

Observed SEVIRI 6.2-μm brightness temperatures (K) valid at 1800 UTC (a) 16 May, (b) 17 May, (c) 18 May, (d) 19 May, and (e) 20 May 2014.

Citation: Monthly Weather Review 146, 1; 10.1175/MWR-D-17-0171.1

• Download Figure
• Download figure as PowerPoint slide

a. Univariate bias correction results

To explore the ability of individual predictors to remove the bias from all-sky infrared observations, univariate NBC experiments were performed using the satellite zenith angle and various predictors sensitive to clouds and WV, such as the brightness temperature, cloud-top height, and integrated water content over some vertical layer. This section presents results from a subset of these experiments that remove the bias from all-sky SEVIRI 6.2-μm observations. The impact of each predictor is assessed using OMB departure distributions normalized by the standard deviation in a given sample and with 2D probability distributions of the departures plotted as a function of a given predictor. The results are evaluated separately for the original departure distribution and for distributions for which the bias has been removed using either a zeroth- (constant), first- (linear), second- (quadratic), or third-order (cubic) Taylor series polynomial expansion.

1) Observed brightness temperature predictor

As shown by the probability distributions in Fig. 2, the observed 6.2-μm brightness temperatures are an excellent predictor of their own bias, especially when higher-order Taylor series terms are used. The horizontal magenta line in each panel depicts the mean bias of the entire distribution, whereas the shorter horizontal black lines depict the conditional bias in each column and will be used to assess how the bias varies as a function of the predictor value. This terminology is being used to differentiate biases conditioned on the predictor value from the bias of the overall distribution. For example, though each distribution except for the original distribution will have zero overall bias, this obscures the fact that the conditional bias could potentially vary as a function of the predictor value. Inspection of Fig. 2a reveals a nonlinear pattern in the conditional biases, with a tendency for the simulated brightness temperatures to be too warm (cold) when the observed brightness temperatures are colder (warmer) than 235 K. Though the mean bias of the distribution is relatively small (−0.83 K), the nonlinear pattern in the conditional biases means that constant and linear BC terms alone will be unable to remove all of the bias. For example, even though the constant BC term removes the mean bias from the distribution (Fig. 2b), its shape remains the same and, therefore, large conditional biases remain throughout the distribution. Likewise, the first-order BC term removes the linear departure component by raising (lowering) the cold (warm) end of the distribution, which reduces the conditional biases for the coldest brightness temperatures, but turns a positive bias into a negative bias for the warmest brightness temperatures (Fig. 2c). Removal of the constant and linear bias components exposes an asymmetric arch shape in the conditional biases that is largely removed when the second-order quadratic term is used (Fig. 2d), except for nonzero biases that remain at the cold and warm ends of the distribution. Finally, when the third-order cubic term is used, the general shape of the distribution is unchanged; however, it is evident that subtle improvements were made to it given that most of the conditional biases are now close to zero. Together, these results show that even though each BC distribution has zero mean bias, the conditional biases in the distribution are much smaller when the higher-order, nonlinear BC terms are applied to the observation departures.

View Full Size

Probability distributions of 6.2-μm observation-minus-background departures plotted as a function of the observed 6.2-μm brightness temperatures (K) for the (a) original data, and the (b) constant, (c) first-order, (d) second-order, and (e) third-order bias-corrected observations when the observed 6.2-μm brightness temperature is used as the predictor. The horizontal black line segments represent the conditional bias in each column. Data were accumulated at hourly intervals during a 108-h period from 1300 UTC 16 May to 0000 UTC 20 May 2014.

Citation: Monthly Weather Review 146, 1; 10.1175/MWR-D-17-0171.1

• Download Figure
• Download figure as PowerPoint slide

Normalized OMB departure histograms computed using the original observations and the constant, first-, second-, and third-order BC observations are shown in Figs. 3a–e. Each histogram is normalized based on its variance, with the curved red line on each panel representing a Gaussian distribution with zero mean and a variance equal to that of the sample. Overall, the variance and root-mean-square error (RMSE) are greatly reduced when the first-order BC coefficients are applied to the observations (Fig. 3c), which is primarily due to the smaller departures for the colder brightness temperatures (e.g., Fig. 2c). The variance was further reduced when the second-order BC was used, with only minimal changes occurring when this was expanded to a third-order BC (Figs. 3d,e). The fact that the higher-order terms only had a small impact on these statistics while simultaneously having a large positive impact on the conditional biases in Fig. 2 illustrates that more detailed analysis methods such as 2D probability distributions can provide additional insight into the characteristics of the OMB departure distributions. Comparison of the histograms also shows that the negative skewness in the original distribution (Fig. 3a) changes to positive skewness after the BC terms are applied. This behavior primarily results from a conditional positive skewness for brightness temperatures 230 K that is evident in Fig. 2a by the tendency for the conditional bias in each column to be located above the bin with the maximum probability. Because the same BC is applied to a given brightness temperature regardless of its OMB departure, the positive skewness in the conditional distributions is preserved as they are shifted upward, thereby leading to a positive skewness in the full BC distributions.

View Full Size

Probability density function of normalized 6.2-μm observation-minus-background departures for the (a) original and (b) constant bias correction distributions. The corresponding first-, second-, and third-order bias correction error distributions when the (c)–(e) observed 6.2-μm brightness temperatures, (f)–(h) NWC SAF cloud-top heights, (i)–(k) model-simulated total integrated water content (IWC) in the 100–700-hPa layer, (l)–(n) satellite zenith angle, or (o)–(q) observed 6.2-μm brightness temperatures, satellite zenith angle, and IWC are used as the predictors are also shown. Data were accumulated at hourly intervals during a 108-h period from 1300 UTC 16 May to 0000 UTC 20 May 2014.

Citation: Monthly Weather Review 146, 1; 10.1175/MWR-D-17-0171.1

• Download Figure
• Download figure as PowerPoint slide

2) Cloud-top height predictor

Because infrared observations are very sensitive to the vertical distribution of clouds, an experiment was performed using the NWC SAF cloud-top height retrievals as the BC predictor to better isolate the impact of clouds. To provide complete domain coverage, the clear-sky observations were assigned a height equal to the model terrain elevation. Overall, the conditional biases in the original distribution (Fig. 4a) are close to zero for cloud-top heights 7 km; however, the biases increase for clouds above this level and peak near −6 K for cloud-top heights 10 km. This is a complex error pattern that a constant BC scheme is unable to fix (Fig. 4b). Indeed, the upward shift of the distribution to remove the mean bias actually worsens the conditional biases for cloud-top heights 7 km, while leading to only minor improvements for the upper-level clouds. The linear correction (Fig. 4c) slightly improves the conditional biases for lower- and upper-level clouds, but worsens the bias for midlevel clouds, which together slightly reduces the variance in the overall distribution (Fig. 3f). Use of the second-order quadratic term substantially improves the distribution by removing the arch in the conditional bias pattern by decreasing the magnitude of the positive (negative) OMB departures for cloud tops located in the middle (upper) troposphere (Fig. 4d). These changes resulted in a much smaller variance in the histogram (Fig. 3g). As was the case in the previous section, the third-order BC led to slightly smaller conditional biases across most of the distribution (Fig. 4e), but had minimal impact on the statistics of the overall distribution (Fig. 3h). Though the cloud-top height predictor was unable to reduce the variance of the full distribution as much as the brightness temperature predictor did, the NBC method was still able to greatly improve the distribution by decreasing the conditional biases. Its use also led to a more symmetric OMB departure distribution (Fig. 3h). These results show that cloud-top height information can be used to remove the bias from all-sky infrared observations if higher-order Taylor series terms are used.

View Full Size

As in Fig. 2, but for probability distributions plotted as a function of the NWC SAF cloud-top height retrieval (km) when this quantity is also used as the BC predictor.

Citation: Monthly Weather Review 146, 1; 10.1175/MWR-D-17-0171.1

• Download Figure
• Download figure as PowerPoint slide

3) Vertically integrated water content predictor

In this section, the impact of using a BC predictor that depicts the total water content over a vertical layer is assessed. Numerous experiments were performed using different vertical layers; however, for brevity, results are only shown for the predictor that encapsulates the total water content between 100 and 700 hPa because that is the portion of the atmosphere where 6.2-μm brightness temperatures are most sensitive. Unlike the previous predictors, this predictor is computed using model output. The total water content is calculated for each ensemble member by converting the WV and all cloud hydrometeor mixing ratios in each model layer into millimeters and then integrating over the 100–700-hPa layer. Inspection of Fig. 5a shows that this predictor has a less complex OMB departure pattern than occurred when the cloud-top height and brightness temperatures were used as the predictors. There are, however, slightly larger biases on both ends of the distribution, with a small upward slope in the maximum probabilities as the total water content increases. This linear error trend is removed by the linear BC term (Fig. 5c), which reduces the conditional biases when the total water content is 7 mm, but increases it elsewhere. The subtle arch in the conditional biases is subsequently removed after applying the second-order quadratic term (Fig. 5d), with only minor changes occurring after the third-order term is used (Fig. 5e). Comparison of the histograms (Figs. 3i–k) shows that the total water predictor had only a small impact on the variance of the full distribution; however, the scatterplots showed that it still reduced the conditional bias across most of the distribution. Even so, this predictor still had a much smaller impact than the previous predictors that were directly sensitive to the cloud-top height, which indicates that the location of the cloud top rather than the vertically integrated cloud and WV content is a more effective BC predictor for all-sky infrared brightness temperatures.

View Full Size

As in Fig. 2, but for probability distributions plotted as a function of the vertically integrated total water content (mm) over the 100–700-hPa layer when this quantity is also used as the BC predictor.

Citation: Monthly Weather Review 146, 1; 10.1175/MWR-D-17-0171.1

• Download Figure
• Download figure as PowerPoint slide

4) Satellite zenith angle predictor

Given that the satellite zenith angle is widely used in operational BC methods, an additional experiment was performed using it as the BC predictor. After adjusting for the mean bias in the original distribution, the conditional biases are close to zero across the entire distribution, with only a slight downward trend in the bias for zenith angles 48° (Fig. 6b). Application of the first- to third-order BC terms (Figs. 6c–e) eliminated most of these conditional biases; however, the impact of this predictor on the statistics of the entire distribution was negligible according to the histograms (Figs. 3l–n). These results indicate that the bias in the observations is only very weakly related to the satellite zenith angle; however, the small improvements made to the conditional biases by the second- to third-order terms also show that there is a small nonlinear bias component that can be removed when using this predictor.

View Full Size

As in Fig. 2, but for probability distributions plotted as a function of the satellite zenith angle (°) when this quantity is also used as the BC predictor.

Citation: Monthly Weather Review 146, 1; 10.1175/MWR-D-17-0171.1

• Download Figure
• Download figure as PowerPoint slide

b. Clear- and cloudy-sky error evaluation

Next, the relative impact of the linear and nonlinear BC terms on the clear and cloudy-sky observations is examined more closely using a subset of the 6.2-μm brightness temperatures for which both the model background and a given observation were identified as being clear or cloudy. Each observation was classified as clear or cloudy based on the NWC SAF cloud mask dataset whereas each model grid point was deemed to be clear (cloudy) if the sum of all cloud hydrometeor mixing ratios over the entire vertical profile was less (greater) than kg . The 2D probability distributions for the clear-sky matched observations are shown in Fig. 7, with the corresponding histograms shown in Fig. 8. The observed 6.2-μm brightness temperatures were used as the BC predictor. Inspection of Fig. 7a reveals that the original distribution contains both a systematic bias and a large linear trend where mostly negative OMB departures for the colder brightness temperatures transition into mostly positive departures for the warmer brightness temperatures. The linear trend indicates that the WV field in the model background is more uniform than observed such that the model tends to be too wet (dry) in regions where the observations indicate less (more) WV. Overall, most of the bias is removed from the clear-sky observation departures using only the constant and first-order terms, with little or no impact due to the higher-order terms (Figs. 7b–e). This behavior is consistent with existing BC schemes that use constant and linear corrections to remove the bias from clear-sky observation departures.

View Full Size

As in Fig. 2, but for probability distributions for clear-sky matched observations plotted as a function of the observed brightness temperature (K) when this quantity is also used as the BC predictor.

Citation: Monthly Weather Review 146, 1; 10.1175/MWR-D-17-0171.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Probability density function of normalized clear-sky matched 6.2-μm observation-minus-background departures for the (a) original data, and the (b) constant, (c) first-order, (d) second-order, and (e) third-order bias corrected observations when the observed 6.2-μm brightness temperature is used as the predictor. Data were accumulated at hourly intervals during a 108-h period from 1300 UTC 16 May to 0000 UTC 20 May 2014.

Citation: Monthly Weather Review 146, 1; 10.1175/MWR-D-17-0171.1

• Download Figure
• Download figure as PowerPoint slide

For the cloud-matched observations shown in Figs. 9 and 10, the NWC SAF cloud-top height retrievals were used as the predictor. The OMB departure pattern and conditional biases for these observations are very similar to that shown in Fig. 4 when both clear- and cloudy-sky observations were included in the regression. This includes the generally positive departures for midlevel clouds and the transition to large negative departures for the upper-level clouds (Fig. 9a). Large departures remained in the distribution for all cloud-top heights after the constant and linear BC terms were applied to the observations (Fig. 9c). It is only when the second- and third-order terms are used that the conditional biases become close to zero throughout the entire distribution (Figs. 9d,e). The histograms in Fig. 10 also reveal that the quadratic and cubic terms had a much larger impact on the overall statistics than occurred for the clear-sky matched observations. These results provide further evidence that the nonlinear conditional biases evident in the all-sky scatterplots in section 4a primarily result from biases associated with the cloudy observations. It also shows that the NBC method is an effective method to remove both linear and nonlinear biases from all-sky infrared brightness temperature departures if a suitable cloud-sensitive variable is used as the predictor.

View Full Size

As in Fig. 2, but for probability distributions for cloudy-sky matched observations plotted as a function of the NWC SAF cloud-top height retrieval (km) when this quantity is also used as the BC predictor.

Citation: Monthly Weather Review 146, 1; 10.1175/MWR-D-17-0171.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Probability density function of normalized cloudy-sky matched 6.2-μm observation-minus-background departures for the (a) original data, and the (b) constant, (c) first-order, (d) second-order, and (e) third-order bias corrected observations when the NWC SAF cloud-top height retrieval is used as the predictor. Data were accumulated at hourly intervals during a 108-h period from 1300 UTC 16 May to 0000 UTC 20 May 2014.

Citation: Monthly Weather Review 146, 1; 10.1175/MWR-D-17-0171.1

• Download Figure
• Download figure as PowerPoint slide

c. Multivariate bias correction results

In addition to the univariate NBC experiments discussed in previous sections, multivariate experiments were performed to assess the impact of using more than one predictor to remove the observation bias. For a third-order polynomial expansion using two variables, it is necessary to solve for seven coefficients in Eq. (11), whereas 22 coefficients are computed when three predictors are used. Because a direct approach is used to simultaneously estimate all of the BC coefficients, it is not possible to determine the individual contribution of each predictor on the OMB departures; however, the total contribution of all of the predictors within a given Taylor series order (e.g., first, second, and third) can still be inferred through comparison of the results obtained using different order expansions. Though using more than one variable greatly increases the size of the matrix, it is still computationally efficient to solve for the inverse of given its small dimensions.

Numerous experiments using different predictor combinations and a second- or third-order polynomial expansion were performed; however, for brevity, this section only includes results from the combination that had the largest impact on the OMB departure distributions. This particular configuration employed a third-order expansion with the satellite zenith angle, 100–700-hPa total water content, and observed brightness temperatures for a given satellite band used as the BC predictors for that band. A separate multivariate experiment (not shown) that employed the cloud-top height rather than the brightness temperature as the third predictor revealed that it had a smaller impact, similar to what occurred with the univariate experiments shown earlier. There may be some overlap between the brightness temperature and satellite zenith angle predictors; however, this should be minimal because the zenith angle predictor primarily accounts for potential biases in the radiative transfer model associated with the pathlength through the atmosphere, whereas the brightness temperature predictor is being used as a proxy for the cloud-top height given its strong sensitivity to the cloud top. Unlike the previous sections that focused exclusively on the 6.2-μm band, this section presents results from experiments that removed the bias from both of the SEVIRI WV-sensitive bands (e.g., 6.2 and 7.3 μm). All observations, both clear and cloudy sky, were used during these experiments.

1) SEVIRI 6.2- example

Figure 11 shows the OMB departure distributions for the 6.2-μm multivariate NBC experiment, with the corresponding normalized histograms shown in Figs. 3o–q. Comparison to Fig. 2 shows that the departure distributions for the multivariate case are similar to those from the univariate case employing only the observed brightness temperature as the BC predictor. This is not surprising given that the experiments employing the satellite zenith angle and total water content predictors both had a much smaller impact on the distributions (Figs. 5 and 6). Overall, the shape of the distribution is improved after the linear term is used; however, there are still large conditional biases at both ends of the distribution (Fig. 11c). The arch pattern in the conditional bias was subsequently removed after the quadratic term was applied (Fig. 11d), with slightly smaller (larger) biases occurring at the warm (cold) end of the distribution after using the third-order cubic term (Fig. 11e). Though the distributions are similar to those shown in Fig. 2, it is evident that the width of the conditional distribution is less for all predictor values. This is encouraging because it shows that even though the impact of the satellite zenith angle and total water content predictors was relatively small when used individually, they still provided new information that further reduced the OMB departures when used in combination with the observed brightness temperature predictor. Inspection of the histograms (Figs. 3o–q) shows that the variance was greatly reduced compared to the univariate experiments; however, each of the distributions had a large positive skewness similar to that seen in Figs. 3c–e when the brightness temperature was used as the BC predictor. It is important to note, however, that quality control measures could potentially be used to reduce the skewness in the distribution after the BC terms are applied. This topic will be explored in a future study.

View Full Size

As in Fig. 2, but for probability distributions plotted as a function of the observed 6.2-μm brightness temperatures when the observed 6.2-μm brightness temperature, satellite zenith angle, and vertically integrated total water content from 100 to 700 hPa are used as the BC predictors.

Citation: Monthly Weather Review 146, 1; 10.1175/MWR-D-17-0171.1

• Download Figure
• Download figure as PowerPoint slide

2) SEVIRI 7.3- example

In this section, we assess the ability of the multivariate NBC method to improve the observation error characteristics of the 7.3-μm band. As discussed in section 2a, observations from this band are sensitive to WV and clouds in the middle and upper troposphere, with a weighting function that peaks near 500 hPa in clear-sky scenes. Overall, each of the OMB departure distributions (Fig. 12) have shapes that are similar to the corresponding 6.2-μm distributions (Fig. 11); however, their error range is larger because the weighting function for this band peaks at a lower level in the troposphere, thereby leading to potentially larger departures because of mismatched clouds in the observations and model background. Though the linear BC term substantially improves the distribution by making the departures less negative for colder brightness temperatures, nonzero conditional biases remain across most of the distribution, with negative (positive) biases occurring for brightness temperatures colder (warmer) than 230 K (Fig. 12c). As occurred in the previous experiments, the conditional biases are almost eliminated after the second-order BC term is used, with minimal changes occurring because of the third-order term (Figs. 12d, e). The negative skewness present in the original histogram (Fig. 13a) switches to a large positive skewness after the linear BC term is used (Fig. 13c). Inspection of the OMB departure distributions shows that the positive skewness developed in response to the large upward shift in the conditional distributions for the colder brightness temperatures (Fig. 12a) that exposed the conditional positive skewness in the original distribution for warmer brightness temperatures that was being masked in the overall histogram by the large negative OMB departures. Another notable feature of the histograms is that their peaks are higher and narrower than the 6.2-μm histograms (Figs. 3o–q). This strongly non-Gaussian behavior was already present in the original histogram and is likely due to the large percentage of clear-sky observations containing small departures combined with fatter tails due to cloud displacement errors. Even so, these results show that the NBC method improved the distribution such that the variance was much lower and the conditional biases were reduced to near zero across most of the distribution. Also, as was the case with the 6.2-μm band, the linear BC term had the largest impact on the overall statistics; however, the variance was also reduced when using the higher-order nonlinear BC terms.

View Full Size

Probability distributions of 7.3-μm observation-minus-background departures plotted as a function of the observed 7.3-μm brightness temperatures (K) for the (a) original data, and the (b) constant, (c) first-order, (d) second-order, and (e) third-order bias corrected observations when the observed 7.3-μm brightness temperature, satellite zenith angle, and model-integrated total water content from 100 to 700 hPa are used as the predictors. Data were accumulated at hourly intervals during a 108-h period from 1300 UTC 16 May to 0000 UTC 20 May 2014.

Citation: Monthly Weather Review 146, 1; 10.1175/MWR-D-17-0171.1

• Download Figure
• Download figure as PowerPoint slide

View Full Size

Probability density function of normalized 7.3-μm observation-minus-background departures for the (a) original data, and the (b) constant, (c) first-order, (d) second-order, and (e) third-order bias-corrected observations when the observed 7.3-μm brightness temperatures, satellite zenith angle, and vertically integrated total water content from 100 to 700 hPa are used as the BC predictors. Data were accumulated at hourly intervals during a 108-h period from 1300 UTC 16 May to 0000 UTC 20 May 2014.

Citation: Monthly Weather Review 146, 1; 10.1175/MWR-D-17-0171.1

• Download Figure
• Download figure as PowerPoint slide

5. Discussion and conclusions

In this study, output from a high-resolution, regional-scale ensemble DA system was used to explore the ability of an innovative method to remove the bias associated with all-sky satellite infrared brightness temperatures using a Taylor series polynomial expansion of the OMB departures. This so-called NBC method uses OMB statistics accumulated over some period of time to remove linear and nonlinear conditional biases in a distribution through use of higher-order Taylor series terms and a set of BC predictors. Nonlinear conditional biases can be identified using second- (quadratic) and third-order (cubic) terms (and even higher-order terms if desired), whereas the constant and linear bias components can be diagnosed using the zeroth- and first-order terms, respectively.

The ability of the NBC method to effectively remove the bias associated with all-sky SEVIRI infrared brightness temperatures was assessed using output from high-resolution ensemble DA experiments performed using the KENDA system. OMB departure statistics for the 6.2- and 7.3-μm bands sensitive to clouds and WV in the upper and middle troposphere, respectively, were accumulated at hourly intervals during a 108-h period from 16 to 21 May 2014 using output from the COSMO-DE domain that covers Germany and surrounding areas with 2.8-km horizontal grid spacing. Conventional observations were actively assimilated, whereas the SEVIRI observations were passively monitored and therefore did not affect the analyses during the hourly assimilation cycles. Model-equivalent brightness temperatures were computed for each observation and ensemble member using the RTTOV radiative transfer model. The study period contained both clear-sky areas and a wide range of cloud types that together promoted a realistic assessment of the NBC method during the warm season.

Univariate and multivariate NBC experiments were performed using the satellite zenith angle and other predictors sensitive to clouds and WV, with their impact on the conditional bias and other aspects of the OMB departure distributions assessed using normalized histograms and probability distributions plotted as a function of the predictor. Overall, the results revealed that there are often strongly nonlinear conditional bias patterns in the OMB probability distributions that cannot be removed using only constant and linear BC terms. Though the overall bias of each distribution is equal to zero regardless of the order of the Taylor series expansion, there are often large conditional biases that vary as a function of the BC predictor. Because each SEVIRI band had a relatively small systematic bias, the constant BC term only had a small impact on the distributions. The linear first-order term generally had the largest impact on the statistics of the entire distribution as measured by reductions in the variance; however, conditional biases often remained across much of the distribution. These conditional biases were typically reduced to near zero across the entire distribution only after the nonlinear second- and third-order terms were applied to the OMB departures. Indeed, the conditional bias patterns often exhibited an arch shape for which the second-order quadratic term is ideally suited to remove. The tendency for the nonlinear terms to have a small impact on the variance of the entire distribution while simultaneously having a large positive impact on the conditional biases also illustrates that detailed analysis methods such as 2D probability distributions provide valuable insight into the behavior of the BC method that is not possible using traditional 1D error histograms.

Inspection of the univariate NBC results showed that the variance of the BC distributions was smallest when the brightness temperature observations were used as the BC predictor. The variance was also substantially reduced when the NWC SAF cloud-top height retrievals were used as the predictor. Both of these predictors were able to diagnose and remove nonlinear biases associated with the cloudy observations. For example, large positive conditional biases for midlevel clouds transitioned into large negative conditional biases for upper-level clouds. Though not examined during this study, the different signs of the conditional biases for these clouds could be related to the ability of the COSMO Model and RTTOV to properly simulate ice and mixed-phase cloud properties. The experiments using the satellite zenith angle or vertically integrated water content showed that these BC predictors had a much smaller impact on the variance of the overall distribution. This behavior indicates that variables sensitive to the cloud-top height are more effective BC predictors for all-sky infrared brightness temperatures, especially when higher-order Taylor series terms are included. Even so, the multivariate experiments showed that though the zenith angle and total water content predictors only had a relatively small impact on the departure histograms when used individually, they still provided new information that greatly reduced the variance of the distribution when used in combination with the observed brightness temperature predictor.

Additional univariate NBC experiments were performed to examine the influence of linear and nonlinear components on the OMB departure distributions for clear- and cloudy-sky observations using a subset of the 6.2-μm brightness temperatures for which both a given observation and the corresponding model grid point were identified as being clear or cloudy. Overall, comparisons of the statistics for the clear-sky and cloudy-sky matched observations revealed that nonlinear error sources are much more important for cloudy-sky observations as signified by the much larger impact of the second- and third-order Taylor series terms on the variance and the conditional biases in the distributions. For the clear-sky observations, the conditional biases could be effectively removed using only the zeroth- and first-order terms, which is consistent with existing operational BC methods that typically remove the bias from clear-sky satellite observations using a set of constant and linear BC coefficients. These results show that the nonlinear conditional bias patterns evident in the all-sky OMB departure distributions primarily resulted from nonlinear biases in the cloudy-sky infrared brightness temperatures. They also show that the NBC method can effectively remove both linear and nonlinear conditional biases from all-sky infrared brightness temperatures provided that a suitable cloud-sensitive variable is used as one of the predictors.

Future work includes running cycled DA experiments using the KENDA system to assess the impact of the NBC method on the forecast accuracy when assimilating clear- and cloudy-sky infrared brightness temperatures. Additional experiments will be necessary to explore the ability of the method to remove biases from the OMB departures when the simulated brightness temperatures and cloud-top heights are used as the BC predictors rather than their observed counterparts. Preliminary results indicate that predictors derived from the NWP model cloud field rather than the observations have a smaller impact on the overall statistics as measured by reductions in variance; however, they were still able to effectively remove the conditional biases across most of the distribution when higher-order Taylor series terms were used. These results also indicate that it may be necessary to use up to a fourth-order polynomial to remove the bias if the NWP-derived quantities are used rather than their observed counterparts. A more detailed assessment of this sensitivity is currently under way. Additional experiments will also be necessary to explore the ability of the NBC method to remove biases from infrared bands that are sensitive to the land surface or other atmospheric constituents such as ozone, as well as for all-sky microwave and visible radiances.

Though the NBC method used in this paper was implemented as a static, offline method, it could also be incorporated into online methods such as VarBC through inclusion of additional nonlinear predictors. For example, the VarBC system at the Met Office uses Legendre polynomial predictors to remove residual scan biases and Fourier predictors to correct complex orbital biases in some satellite sensors (Cameron and Bell 2016). Higher-order predictors, such as the quadratic form of the temperature lapse rate and fourth-order polynomial of the satellite angle bias, are also widely used in operational VarBC systems. Zhu et al. (2015) recently showed that inclusion of a quadratic aircraft ascent/descent term reduced the bias when assimilating aircraft temperature observations. Results from the current study could be used to help inform the development of operational DA systems as they continue to expand into all-sky satellite DA. Finally, many of the all-sky OMB departure distributions exhibited narrow peaks and fat tails that could potentially be better represented using a Huber norm (Huber 1972) representation, which has been shown to lead to improved quality control and more observations being assimilated (Tavolato and Isaksen 2015). Further research is necessary to determine if using a Huber norm in combination with the NBC method can improve existing quality control methods by identifying erroneous observations after the nonlinear conditional biases have been removed from the distribution. This approach could potentially preserve more cloud-affected observations where nonlinear biases are more prevalent, thereby leading to additional observations being assimilated in sensitive areas of the domain.