• Basili, P., P. Ciotti, and D. Solimini, 1981: Inversion of ground-based radiometric data by Kalman filtering. Radio Sci.,16, 83–91.

    • Crossref
    • Export Citation
  • Decker, M. T., and J. A. Schroeder, 1991: Calibration of ground-based microwave radiometers for atmospheric remote sensing. NOAA Tech. Memo. ERL WPL-197, 16 pp. [Available from National Technical Information Service, 5285 Port Royal Rd., Springfield, VA 22161.].

  • ———, E. R. Westwater, and F. O. Guriaud, 1978: Experimental evaluation of ground-based microwave radiometric sensing of atmospheric temperature and water vapor profiles. J. Appl. Meteor.,17, 89–92.

    • Crossref
    • Export Citation
  • England, M. N., R. A. Ferrare, S. H. Melfi, D. N. Whiteman, and T. A. Clark, 1992: Atmospheric water vapor measurements: Comparison of microwave radiometry and lidar. J. Geophys. Res.,97, 899–916.

    • Crossref
    • Export Citation
  • Ferrare, R. A., S. H. Melfi, D. N. Whiteman, and K. D. Evans, 1992: Raman lidar measurements of Pinatubo aerosols over southeastern Kansas during November–December 1991. Geophys. Res. Lett.,19, 1599–1602.

    • Crossref
    • Export Citation
  • ———, ———, ———, ———, F. J. Schmidlin, and D. O’C. Starr, 1995: A comparison of water vapor measurements made by Raman lidar and radiosondes. J. Atmos. Oceanic Technol.,12, 1178–1195.

    • Crossref
    • Export Citation
  • Gelb, A., 1988: Applied Optimal Estimation. The MIT Press, 374 pp.

  • Han, Y., and E. R. Westwater, 1995: Remote sensing of tropospheric water vapor and cloud liquid water by integrated ground-based sensors. J. Atmos. Oceanic Technol.,12, 1050–1059.

    • Crossref
    • Export Citation
  • ———, J. B. Snider, E. R. Westwater, S. H. Melfi, and R. A. Ferrare, 1994: Observations of water vapor by ground-based microwave radiometers and Raman lidar. J. Geophys. Res.,99, 18 695–18 702.

  • Kaminski, P. G., A. E. Bryson, and S. F. Schmidt, 1971: Discrete square root filtering: A survey of current techniques. IEEE Trans. Autom. Control,AC-16, 727–736.

    • Crossref
    • Export Citation
  • Ledsham, W. H., and D. H. Staelin, 1978: An extended Kalman–Bucy filter for atmospheric temperature profile retrieval with a passive microwave sounder. J. Appl. Meteor.,17, 1023–1033.

    • Crossref
    • Export Citation
  • Martner, B. E., and Coauthors, 1993: An evaluation of wind profiler, RASS, and microwave radiometer performance. Bull Amer. Meteor. Soc.,74, 599–613.

    • Crossref
    • Export Citation
  • Melfi, S. H., and D. H. Whiteman, 1985: Observation of lower atmospheric moisture structure and its evolution using a Raman lidar. Bull. Amer. Meteor. Soc.,66, 1288–1292.

    • Crossref
    • Export Citation
  • ———, ———, and R. A. Ferrare, 1989: Observation of atmospheric fronts using Raman lidar moisture measurements. J. Appl. Meteor.,28, 789–806.

    • Crossref
    • Export Citation
  • Rodgers, C. D., 1976: Retrieval of atmospheric temperature and composition from remote measurements of thermal radiation. Rev. Geophys. Space Phys.,14, 609–624.

    • Crossref
    • Export Citation
  • Ruf, C. S., and C. T. Swift, 1988: Atmospheric profiling of water vapor density with a 20.5–23.5 GHz autocorrelation radiometer. J. Atmos. Oceanic Technol.,5, 539–546.

    • Crossref
    • Export Citation
  • Skoog, B. G., J. I. H. Askne, and G. Elgered, 1982: Experimental determination of water vapor profiles from ground-based radiometer measurements at 21.0 and 31.4 GHz. J. Appl. Meteor.,21, 394–400.

  • Stankov, B. B., B. E. Martner, and M. K. Politovich, 1995: Moisture profiling of the cloudy winter atmosphere using combined remote sensors. J. Atmos. Oceanic Technol.,12, 488–510.

    • Crossref
    • Export Citation
  • Strang, G., 1988: Linear Algebra and Its Applications. Academic, 528 pp.

  • Wang, J. R., and Coauthors, 1983: Profiling atmospheric water vapor by microwave radiometry. J. Climate Appl. Meteor.,22, 779–788.

    • Crossref
    • Export Citation
  • Westwater, E. R., 1978: The accuracy of water vapor and cloud liquid determination by dual-frequency ground-based microwave radiometry. Radio Sci.,13, 677–685.

    • Crossref
    • Export Citation
  • ———, M. T. Decker, A. Zachs, and K. S. Gage, 1983: Ground-based remote sensing of temperature profiles by a combination of microwave radiometry and radar. J. Climate Appl. Meteor.,22, 126–133.

    • Crossref
    • Export Citation
  • Whiteman, D. N., S. H. Melfi, and R. A. Ferrare, 1992: Raman lidar system for the measurement of water vapor and aerosols in earth’s atmosphere. Appl. Opt.,31, 3068–3082.

    • Crossref
    • Export Citation
  • View in gallery

    Fig. 1. Comparison of transition errors computed using a 3-h radiosonde database (solid line) and a set of lidar data with a 2-min temporal resolution (dashed line).

  • View in gallery

    Fig. 2. Water vapor mixing ratio profile retrievals every half hour. Solid line—Raman lidar measurements; dashed line—retrievals; dotted line—cloud-base height. Data were collected during the FIRE II experiment.

  • View in gallery

    Fig. 3. Time series of cloud-base height identified from aerosol profiles measured by the lidar. Examples of water vapor profile retrievals during this period are shown in Fig. 2.

  • View in gallery

    Fig. 4. (a) An example of retrieval using the technique described in the text. Dash–dot line—retrieval; solid line—ground truth (from lidar); dotted line—artificial cloud base. Data were collected during FIRE II. The cloud base is created by artificially extending existing cloud base beyond clear periods. The lidar sounding is truncated at the artificial cloud base and used as cloudy measurements. The original lidar sounding is used as ground truth. (b) An example of retrieval when lidar provides no historic measurements for the portion above the cloud base. Same notation as in (a).

  • View in gallery

    Fig. 5. Time series of estimated retrieval errors. Solid line—estimated errors of the two-stage retrieval algorithm; dashed line—estimated errors of the Kalman performed at the first stage; dotted line—uncertainties of the statistical constraint.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 113 108 4
PDF Downloads 22 17 1

Applications of Kalman Filtering to Derive Water Vapor Profiles from Raman Lidar and Microwave Radiometers

View More View Less
  • 1 University of Colorado/Cooperative Institute for Research in Environmental Sciences and NOAA/Environmental Technology Laboratory, Boulder, Colorado
  • | 2 Hughes-STX Corporation, Lanham, Maryland
© Get Permissions
Full access

Abstract

A two-stage retrieval technique is presented for deriving water vapor profiles from data provided by a Raman lidar, a microwave radiometer, a radio acoustic sounding system, and surface in situ instruments. In the first stage, a Kalman filtering algorithm is applied to derive water vapor profiles using surface in situ and current and past Raman measurements. In the second stage, a statistical inversion technique is applied to combine the Kalman retrieval with radiometric and climatological data. This retrieval method is tested using data collected during the First ISCCP (International Satellite Cloud Climatology Project) Regional Experiment II experiment. The method is demonstrated to provide accurate profiles at altitudes above which the Raman lidar technique is limited.

Corresponding author address: Yong Han, NOAA/ERL/ETL, 325 Broadway, Boulder, CO 80303.

Email: yhan@etl.noaa.gov

Abstract

A two-stage retrieval technique is presented for deriving water vapor profiles from data provided by a Raman lidar, a microwave radiometer, a radio acoustic sounding system, and surface in situ instruments. In the first stage, a Kalman filtering algorithm is applied to derive water vapor profiles using surface in situ and current and past Raman measurements. In the second stage, a statistical inversion technique is applied to combine the Kalman retrieval with radiometric and climatological data. This retrieval method is tested using data collected during the First ISCCP (International Satellite Cloud Climatology Project) Regional Experiment II experiment. The method is demonstrated to provide accurate profiles at altitudes above which the Raman lidar technique is limited.

Corresponding author address: Yong Han, NOAA/ERL/ETL, 325 Broadway, Boulder, CO 80303.

Email: yhan@etl.noaa.gov

1. Introduction

In ground-based remote sensing of atmospheric water vapor, Raman lidars and microwave radiometers have been used in routine operations and field investigations. Microwave radiometers have been primarily used to measure integrated water vapor and cloud liquid. The most significant advantage in microwave radiometry is the capability to measure water vapor and liquid water in almost all nonprecipitating weather conditions. The ability to profile water vapor by microwave radiometers has also been investigated theoretically and experimentally (Decker et al. 1978; Skoog et al. 1982; Ruf and Swift 1988). Because of insufficient profile structure information in the measurements themselves, retrieving accurate profiles requires application of constraints. Raman lidars have the capability to profile water vapor up to 7 km or higher with vertical resolutions of better than 100 m (Melfi and Whiteman 1985; Melfi et al. 1989) in cloud-free and nighttime environments. Under cloudy conditions, the lidar beams are attenuated by clouds and usually cannot probe through liquid clouds. During daytime conditions, contaminations by sunlight limit lidar measurements to the lowest 3 or 4 km. Because of the desire to have continuous measurements of water vapor and because of the weaknesses inherent in each of these remote sensing systems, large field experiments, such as those at the Department of Energy’s (DOE’s) Cloud And Radiation Testbed (CART) Central Facility in Lamont, Oklahoma, will deploy routinely both radiometers and Raman lidars, among with other remote sensing systems.

An approach to increase our capability in water vapor profiling is to combine pieces of information gathered from various remote sensors. Such an approach has been applied to radiometer-based temperature and water vapor inversion (Westwater et al. 1983; Stankov et al. 1995; Han and Westwater 1995). These applications showed that radiometric profile inversions could be significantly improved by incorporating additional information from other remote sensing instruments, especially active sensors. In this paper, we explore the feasibility of profiling water vapor by integrating data from a Raman lidar and a microwave radiometer, as well as a radio acoustic sounding system (RASS) and surface in situ instruments. The idea behind this investigation is that the microwave radiometer’s “all-weather” measuring capability may compensate the limitation of a Raman lidar during cloudy and daytime conditions and the Raman lidar’s fine vertical resolution may compensate the coarse vertical resolutions of a microwave radiometer. A two-stage retrieval algorithm is developed to combine the pieces of information from these sensors. In the first stage, a Kalman filtering technique is applied to lidar data alone that incorporates current and past measurements. In the second stage, a statistical inversion method is applied to combine lidar and radiometer measurements using a priori climatological data. We discuss in section 5 our reasons for developing our two-stage retrieval technique. The data collected in November and December 1991 in Coffeyville, Kansas, during the climate experiment First ISCCP (International Satellite Cloud Climatology Project) Regional Experiment II are used for a test of the technique.

2. Instrumentation

During the FIRE II experiment, a substantial number of remote sensors and in situ instruments were operated together. The instruments included here have been described in detail in Han et al. 1994. Here we discuss the aspects that are relevant to this paper.

a. NOAA transportable three-channel microwave radiometer

The NOAA transportable radiometer measures atmospheric radiation at microwave frequencies in units of brightness temperature as a function of elevation angle. The system contains three independent Dicke-type radiometers, operated at 20.6, 31.65, and 90.0 GHz. All three channels respond to water vapor and cloud liquid. The 20.6 GHz channel is sensitive primarily to water vapor, the 31.65 GHz channel is sensitive primarily to liquid water, and the 90.0 GHz channel is sensitive to both vapor and liquid. All three channels have the same beamwidth of 2.5°. Because of anticipated applications at the Department of Energy’s CART site, where only two channel systems are in use, only the two lower-frequency channels were used here. Although, the system is scannable in both elevation and zenith, our observations were in the zenith direction only. Although the minimum temporal resolution is 30 s, a 2-min resolution is adopted here. The absolute calibration of the brightness temperature measurements was done by the so-called tipping curve procedure (Decker and Schroeder 1991). For the FIRE II experiment, 11 separate tip calibrations were performed throughout the 2-month experiment. The absolute accuracy of the two lower-frequency channels is estimated as ±0.5 K at 20.6 GHz and ±0.9 K at 31.65 GHz with 99% confidence limits. The corresponding rms values for the sensitivities, at 2-min averaging time, are 0.03 and 0.02 K, respectively.

The primary derived products of the radiometer are the column-integrated water vapor and liquid water in clouds. In our previous investigation of water vapor and liquid water during the FIRE II experiment (Han et al. 1994), a statistical inversion method was applied to data from all three channels, and the inversion was conditioned as “cloud-free.” Here a physical inversion method is applied (see section 3), which includes the two lower-frequency channels only for both clear and cloudy conditions.

b. Raman lidar

The NASA Goddard Space Flight Center Raman water vapor lidar (Whiteman et al. 1992; Ferrare et al. 1992) uses a XeF laser to transmit light at a wavelength of 351 nm. It receives the backscattered signals from a variety of sources, including aerosol and molecular backscattered light at the laser wavelengths as well as Raman scattered light from water vapor (403 nm), nitrogen (383 nm), and oxygen (372 nm) molecules. In normal operation, data from more than 23 000 pulses are recorded as 1-min profiles with a range resolution of 75 m. Profiles of water vapor mixing ratios are computed from the ratio of the Raman water vapor to Raman nitrogen return signals. During the FIRE II experiment, the profiles were calibrated by using coincident and collocated radiosonde measurements. Assuming Poisson statistics, the standard error associated with the lidar water vapor mixing ratio profiles is represented by the square root of the total number of photon counts. Intercomparisons between Raman lidars, radiosondes, and microwave radiometers have shown discrepancies generally less than 10% among these instruments (Melfi et al. 1989; England et al. 1992; Han et al. 1994; Ferrare et al. 1995). The maximum altitude of the lidar water vapor retrievals under cloud-free conditions is generally determined by the size of the random error, which, in turn, depends on several instrumental and environmental factors (Ferrare et al. 1995). Under nighttime clear sky conditions, profiles up to at least 7 km can be accurately measured. Under daytime conditions, because the measurements are contaminated by background skylight, only the low-altitude portions of water vapor mixing ratio profiles may have significant signal-to-noise ratio. Clouds rapidly attenuate the laser beam so that under cloudy conditions water vapor profiling is generally confined below clouds. The lowest range gate of the lidar during this experiment was 185 m.

In addition to water vapor profiles, profiles of the ratio of the total aerosol to molecular scattering are also measured. The cloud base altitudes, used here, are easily identified from these profiles.

c. Other instruments

A 404.37-MHZ RASS and a set of surface meteorological instruments were collocated with the microwave radiometer and the Raman lidar. During the FIRE II experiment, the RASS profile of virtual temperature ranged from 350 m to an upper height that varied from 1 to 2 km. The associated temporal and vertical resolutions were 15 min and 150 m. The upper range was unusually limited due to problems with winds and radio interference. The major characteristics and the performance of this system were reported by Martner et al. (1993). The accuracy of the virtual temperature measurements is generally considered to be better than 1°C.

Conventional surface meteorological instruments measured near-surface air temperature, humidity, and pressure, with 2-min averaging time. The absolute accuracies of these measurements are estimated as 0.5°C, 5%, and 5 mb for temperature, relative humidity, and pressure.

3. Retrieval algorithm

Excluding of conditions with precipitation, we consider measurements under three different environments: 1) nighttime clear sky, when the water vapor mixing ratio up to 7 km or higher is measured by the lidar; 2) nighttime cloudy sky, when the lidar provides measurements up to cloud base; and 3) daytime clear or cloudy sky, when vapor mixing ratios within a few kilometers above the surface are measured by the lidar. The data, from which water vapor mixing profiles are derived, consist of the measurements by Raman lidar of water vapor mixing ratio profiles and cloud base heights, surface temperature, pressure, and humidity from in situ sensors, integrated water vapor (IWV) from the microwave radiometer, and virtual temperature profiles from RASS.

Let the desired water vapor profile x be an n(x) dimensional vector, x = [x1, x2, . . . , xn(x)]T. Its corresponding vertical coordinate starts at the surface level. The second level is set at the lidar’s first range gate. The remaining levels have adjacent intervals of 75 m, to be consistent with lidar’s range gates. The top level is set at 10 km, a height above which the total amount of water vapor is negligible and that is also above the maximum usable Raman lidar range gate. Under conditions when only a portion of water vapor profile is directly measured, retrieval of x using the dataset described earlier is an ill-posed mathematical problem, and hence, additional information is required for such retrievals. In our retrieval algorithm, we introduce two such sources. One is the information contained in past lidar measurements and the other is statistical information obtained from an a priori water vapor profile ensemble. Our algorithm is a two-stage retrieval. In the first stage, by using past and current surface in situ and lidar measurements, a Kalman filter (KF) technique is applied to derive an n(s) dimensional profile vector s that has the same vertical coordinates as x but less range coverage [n(s) ≤ n(x)]. In the second stage, a statistical inversion method is applied to derive the complete profile x by combining the KF retrieval s with IWV measurements and climatological data. In our original algorithm design, only the first-stage was involved and the statistical constraint was embedded in KF as an initial state. However, the statistical information faded away with the temporal advance of KF. In some situations, a statistical constraint is crucial, and without it large errors may be introduced in the retrievals. For instance, this will occur when there are long periods during which the Raman lidar is limited in height coverage. The two-stage retrieval algorithm always balances the retrievals among the following three data components: current measurements, past measurements, and a priori climatological data. The algorithm is described in the following section.

a. Kalman filtering

Applications of KF to derive atmospheric variables have been reported by Ledsham and Staelin (1978) and Wang et al. (1983) for space-based remote sensing and by Basili et al. (1981) for ground-based sensing. A general description of this technique is given by Gelb (1988). In discrete KF, a state vector si of dimension n(si) describes the state of a system at time step i. A measurement vector zi of dimension n(zi) contains measurements for the system. From zj at all previous time increments (j = 0, 1, . . . , i), KF estimates si. The merit of this technique is that the estimation uses not only current measurements but also a history of measurements. In our application, zi contains water vapor mixing ratio measurements from the surface to the maximum height of the lidar profile, which varies depending on the lidar’s noise level and environmental conditions. Thus, the dimension n(zi) depends on time i. In our system, si and zj use the same vertical coordinates and dimensions n(si) ≥ n(zj) for j = 0, 1,  . . . , i. The dimension n(si) is determined according to the following situations. In the first, the maximum height of current lidar measurements is lower than or equal to that of the maximum of previous measurements; n(si) remains unchanged. In the second, the maximum height of the current measurement is greater than the previous maximum; n(si) is then reset to be equal to the dimension n(zi) of the current measurement vector. In the first situation, a portion of si is not directly measured by the current observation but predicted by KF. Here, the KF is designed mainly for this situation. When the second situation occurs, KF is reinitialized and past measurements are discarded because current measurements have larger range coverage and sufficient accuracy.

The measurement vector zi is linearly related to the state vector si as
ziHisiεi
where Hi is the so-called observation matrix of dimension n(zi) × n(si), εi is the measurement error vector with a covariance C(εi), and the subscript i refers to discrete time. For our case, the matrix Hi has diagonal elements equal to 1 and off-diagonal elements equal to zero. In cases when n(zi) is smaller than n(si), the elements of the last n(si) − n(zi) columns in Hi are set to zero. The first element in εi is the error of surface water vapor mixing ratio and is estimated from the errors in the measurements of surface temperature, pressure, and relative humidity. The remaining elements in εi are the Raman lidar measurement errors, which are related to the statistics of Raman water vapor return signals and given with each lidar vapor mixing ratio profile. In the error covariance C(εi), we assume that the off-diagonal elements are zero, which is equivalent to the assumption that the errors are uncorrelated with each other.
In the KF technique, the state vector si evolves according to a known model with known statistical characteristics of model uncertainties. The evolution model is assumed to be a first-order Markovian process,
siϕi−1si−1wi−1
where the ϕi−1 is the transition matrix and wi−1 is the transition error vector, assumed a zero mean white Gaussian process, independent of si, with a covariance matrix C(wi−1). This equation propagates a state vector s from time i − 1 to time i. In our system, the time interval of the state transition is 2 min. Since the variation of the atmospheric water vapor in 2 min is usually small in comparison with the estimated errors in the retrievals, we approximate the transition by advancing the state vector without modification, which is equivalent to setting the transition matrix to unity. The transition errors may be estimated using a historic profile database. The Raman lidar itself may provide part of such a database if a large number of clear lidar measurements are collected. For this experiment, however, the set of nighttime clear lidar profiles collected during the FIRE II experiment may not be statistically sufficient. The database we used here is a collection of 3-h radiosonde profiles from five observation stations at the DOE CART site during the intensive observation period (IOP) conducted in April and May 1995. Over 700 radiosonde profiles were used for the error estimation. Assuming that the transition errors are time invariant and time uncorrelated, we estimated the errors by calculating the mean-square difference between consecutive 3-h profiles and then dividing the difference by the number of 2-min intervals in the advanced time period. In general, a certain temporal correlation between errors is expected, causing the estimates of the transition errors to differ from those that correctly account for these correlations. As a comparison, we calculated transition errors using data with 2-min resolution collected from 11 nighttime observations by the Raman lidar during the FIRE II experiment. Figure 1 shows the square roots of the diagonal elements in the transition error matrices calculated as a function of height using the two sets of data described above. Below 3 km, the transition errors estimated from the set of 3-h radiosondes are larger than those obtained from 2-min-interval lidar data. Above 5 km, the errors calculated from lidar data increase monotonically with height due to the increasing lidar measurement noise. The inconsistency of the two error profiles may be explained by the differences between the statistical sampling of the two datasets. The lidar data were clear nighttime measurements with 2-min temporal resolution, and the radiosonde data were 3-h soundings during both nighttime and daytime conditions at five stations. Since both sets of transition errors have uncertainties, it is preferable to use the radiosonde-derived transition errors that have larger values than the lidar-derived errors below 3 km. It will be clear later that larger transition errors result in larger uncertainties from the use of past measurements. This effectively gives more weight to current measurements in the profile estimation. Existence of correlation is also inconsistent with the assumption that the transition errors are white noise. However, our testing results have shown that our KF technique under this assumption works well for the data we have examined.
Let ŝi(+) represent the estimate of the state vector at time i and Pŝ,i(+) its error covariance matrix. We estimate ŝi(+) and Pŝ,i(+) from the following procedure. The estimate of the state vector at time i − 1, ŝi−1(+), and its error covariance matrix Pŝ,i−1(+) are propagated to time i according to (2) as
ŝiϕi−1ŝi−1
and
i1520-0426-14-3-480-e4
The ŝi(−) and Pŝ,i(−) are seen as an a priori estimate of si and its error covariance matrix, respectively. On the other hand, from the measurements we can obtain another estimate of si, a minimum length solution ŝm,i (Rodgers 1976; Strang 1988) of (1):
ŝm,iHTiHiHTi−1zi
with an inverse error covariance matrix
P−1ŝmiHTiC−1εiHi
Using the inverse error covariance matrices as weights, ŝi(−) and ŝm,i are combined to yield ŝi(+) as
i1520-0426-14-3-480-e7
with an error covariance matrix Pŝ,i(+) of
i1520-0426-14-3-480-e8
In our computer code, instead of using (7) and (8) directly, equivalent equations (Gelb 1988) are used to improve the computation time for inversions of matrices as
ŝiŝiKiziHiŝi
and
Pŝ,iIKiHiPŝ,i
where I is the unit matrix and Ki is the Kalman gain matrix, calculated as
i1520-0426-14-3-480-e11
In addition, to ensure that the error covariance matrix Pŝ,i(+) will be always positive definite, a square-root technique is applied (Kaminski 1971).

The KF starts at an initial state vector and its error covariance. We use the first arrival of the measurement vector z0 and its error covariance as the initial state. The dimension n(s) of the state vector s is set equal to the dimension n(z0). Then, the KF operates on successive measurement vectors until the arrival of a measurement vector with a dimension larger than the one used. The KF is reinitiated and the dimension of s is reset as discussed earlier. The newest estimate and its covariance are used, on one hand, in the second stage of the retrieval process described in the following, and, on the other hand, in the next process of KF estimation.

b. Statistical inversion

In the second stage, the KF profile estimate ŝ(+) and the IWV derived from the radiometer are combined to form a new measurement vector g of dimension n(g) = n(s) + 1. The IWV, along with integrated cloud liquid (ICL), is derived by using a physical retrieval method (Westwater 1978) as
i1520-0426-14-3-480-e12
and
i1520-0426-14-3-480-e13
where V is the water vapor profile weighted average of the water vapor absorption coefficient; L is the liquid water profile weighted average of cloud coefficients; τ is total optical depth of water vapor and cloud liquid; and “1” and “2” refer to the lower- and upper-frequency channels, respectively. The optical depth τ is derived from the radiometric brightness temperatures Tb,
i1520-0426-14-3-480-e14
where Tmr is the mean radiating temperature, Tbb is the big bang cosmic background with a value of 2.75 K, and τd is the dry air optical depth. The parameters, such as V, L, and Tmr, are calculated using a radiative transfer model from the data provided by our system. The model inputs are pressure, temperature, water vapor, and cloud liquid profiles. They are estimated from the following procedure that is similar to that reported by Han and Westwater (1995). The RASS virtual temperature and lidar water vapor profiles are first statistically extrapolated from their maximum height to the top of the atmosphere. The pressure profiles are then calculated using the hydrostatic equation. Initially, a guess of ICL is distributed moist-adiabatically with height. With these initial profiles the model calculates the parameters in (12)–(14). The quantities IWV and ICL are then retrieved from the brightness temperature measurements. In an iterative process, the initial guess of ICL is replaced by the new derived ICL and a second run is performed to derive IWV and ICL again. At each stage of the iteration, cloud liquid profiles are assumed moist adiabatic with cloud-base heights provided by the lidar. The limiting factors in determining ICL are the accuracies in cloud liquid absorption coefficients and in mean radiating temperatures. Han and Westwater (1995) show that compared with the methods that use climatological means to estimate the parameters in (12)–(14), this iterative method significantly improves the accuracy of ICL by reducing the uncertainties in deriving these cloud absorption coefficients and mean radiating temperatures.
The measurement vector g is related linearly to the profile vector x by an n(g) × n(x) observation matrix A as
gAxεg
where εg is the error vector with an n(g) × n(g) covariance matrix C(εg). In A, the block of the first n(g) − 1 rows has diagonal elements equal to 1 and off-diagonal elements are equal to zero. The n(g)th row, whose elements contain values of dry air density, relates x to IWV. The dry air density profile is calculated from the profiles used for the radiative transfer model discussed earlier. The n(g)th element in εg is the error that combines the estimated error in IWV retrieval and the error in converting water vapor mixing ratio profile to IWV. In C(εg), the block of the first n(g) − 1 rows is Pŝ(+) and the last row has the n(g)th element equal to the error of IWV and the remaining elements equal to zero since we assume the error of IWV is uncorrelated with the errors of ŝ(+).

From a climatological data ensemble of radiosondes from Oklahoma City, Oklahoma, the closest National Weather Service (NWS) radiosonde station to Coffeyville, we obtain an a priori estimate r of the profile x, that is, the mean of the radiosonde ensemble and an error covariance matrix Pr of r. An inverse covariance weighted average using an equation similar to (7) is performed to combine the measurement vector and the climatological data. The result is , the final estimate of the water vapor profile x.

In the nighttime clear-sky cases in which the lidar covers the range close to the top of the retrieval vertical coordinate, the estimate from KF usually yields a much smaller error covariance Pŝ(+) in comparison with the statistical covariance Pr. Hence, the averaging performed in the second stage is in favor of Kalman, resulting in an estimate differing little from that of KF. In other cases in which the lidar measures only a portion of a profile, for the same reason as that in the clear-sky cases, the two-stage retrieval yields a profile with the measured portion similar to the lidar measurements. The other part of the retrieved profile is a result of a balance among past and current measurements and climatological data.

4. Experimental results

During the FIRE II experiment, the Raman lidar observed the atmosphere only during night. There were a total of 14 nights of observations, out of which there were 4 nights when low-level clouds with cloud base at about 2 or 3 km were observed. Unfortunately, during the cloudy periods when the lidar operated, there were no vapor profile observations by other instruments, such as radiosondes. Therefore, a direct comparison of retrievals with “ground truth” is not available. In Fig. 2, we show a 3-h time series of profile retrievals with a half-hour separation; the retrieval is for the time indicated on the plot. The lidar detected clouds right after the first profile in the figure. The cloud base heights gradually moved downward as shown in Fig. 3 and caused the lidar range to decrease with time to a minimum of about 2.5 km at about 0300 UT. Our retrieval technique recovered the full profiles by combining the lidar data with the microwave and climatological data. To compare the retrievals with known profiles, we truncated lidar profiles during clear sky conditions at artificial cloud base heights created by extending existing cloud base to clear periods and used the original lidar soundings as ground truth. Figure 4a shows a retrieval 16 min after the start of the cloudy period. As anticipated, the retrieved profile below the cloud base resembles the lidar measurement, and above the cloud base, the influence of the previous lidar measurements is obvious. Figure 4b shows an example in a situation when there are no historic lidar measurements available above cloud-base height. This situation is likely to happen when clouds persist for a long period or in daytime. The information for the profile above the cloud base is supplied by the climatological data and integrated water vapor, as well as the lidar measurement below the cloud through the correlations characterized by the error covariance matrix. In general, under such conditions, the portion above the lidar profile is smoothed. A suggestion for improving the structure of retrievals is to incorporate radiosonde measurements that are often available during field experiments or from nearby NWS sites and use them in the same way as the lidar measurements.

To see how the retrievals are balanced among current measurements, previous measurements, and climatological data, time series of estimated errors at selected levels are calculated from data collected during FIRE II. An artificial 11-h time series of cloud base heights at 1.2 km was generated and retrievals were performed. Figure 5 shows the square roots of the diagonal elements in the estimated error covariance matrices at 3 km as a function of time. The dashed line shows the estimated errors of KF retrievals, the dotted line the uncertainty of the estimate r from the climatological data, and the solid line estimated errors of x̂. The time series start at a retrieval under clear sky conditions and the remaining retrievals are all under cloudy conditions. It is seen that at the beginning, the error of at 3 km is small because the lidar measurements covered that level. During the cloudy period, water vapor at that level was not measured by the lidar. The influence of the past clear-sky measurements on retrievals depends on the relative magnitudes of the KF transition errors to magnitudes of the uncertainties of the climatological data. The KF retrieval errors grew with time and hence, relatively, the climatological constraint plays an increasingly important role in the second retrieval stage. At about 3 h after the start of clouds, the KF retrieval error has the same magnitude as the uncertainty of the climatological constraint. Thus, near the beginning of the cloudy period, the profile structures above cloud bases are contributed mainly by the lidar measurements and after several hours mainly by the combined information due to the climatological data, the IWV constraint, and the correlation with the lidar-measured portion of the profile. As a result, the estimated errors of remain almost constant about 1 h after the start of clouds.

5. Discussion

Our method combines data from Raman lidar, microwave radiometer, and other sources to derive water vapor profiles under almost all nonprecipitating conditions. It applies a two-stage retrieval algorithm to optimize the use of information contained in past and current measurements and climatological data. We made use of two climatological databases. The first was a high temporal resolution set of 3-h radiosondes that we used to estimate errors associated with our assumptions about the KF transition matrix. The second database was the more conventional climatology associated with 12-h synoptic radiosonde releases in central Oklahoma. The second set provided retrieval coefficients for the radiometer data and conditional constraints on water vapor profiles. Initially, a single stage KF algorithm was used to combine both radiometer and lidar data. However, it was quickly found that when a single high-quality lidar sounding was available, all climatological constraints on the profile were lost and subsequent retrievals could be degraded. This is particularly troublesome when long periods exist when only truncated Raman data are available. To retain the a priori constraints, we developed the two-stage method that was presented in section 3.

We have shown that under nighttime clear sky conditions, retrievals of water vapor profiles differ little from lidar and surface in situ measurements. Under cloudy conditions, the profiles below cloud bases are retrieved mainly from the lidar and surface in situ data. The profiles above the cloud bases are retrieved by balancing the past and current measurements and climatological data. Thus, under cloudy conditions, the profile vertical resolution below cloud bases is determined by the lidar to be 75 m. The vertical resolution above cloud bases is complicated. Near the beginning of cloudy periods, profile structure information is contributed mainly by past clear-sky measurements. Applications of such information are statistically valid but may cause retrieval errors in cases of rapidly changing environments and nonstationary conditions. Several hours after the beginning of cloudy periods, the profile structures are determined mainly by the statistical information and hence, smoothed profile retrievals are expected. In all cases, the profiles are constrained by the integrated water vapor obtained from the microwave radiometer. In addition, a modest extension of the KF technique would also incorporate radiosonde profiles into the retrievals.

Acknowledgments

We thank the reviewers who contributed useful comments to this paper. This work was partially supported by the Environmental Sciences Division of the Department of Energy as a part of their Atmospheric Radiation Measurements Program.

REFERENCES

  • Basili, P., P. Ciotti, and D. Solimini, 1981: Inversion of ground-based radiometric data by Kalman filtering. Radio Sci.,16, 83–91.

    • Crossref
    • Export Citation
  • Decker, M. T., and J. A. Schroeder, 1991: Calibration of ground-based microwave radiometers for atmospheric remote sensing. NOAA Tech. Memo. ERL WPL-197, 16 pp. [Available from National Technical Information Service, 5285 Port Royal Rd., Springfield, VA 22161.].

  • ———, E. R. Westwater, and F. O. Guriaud, 1978: Experimental evaluation of ground-based microwave radiometric sensing of atmospheric temperature and water vapor profiles. J. Appl. Meteor.,17, 89–92.

    • Crossref
    • Export Citation
  • England, M. N., R. A. Ferrare, S. H. Melfi, D. N. Whiteman, and T. A. Clark, 1992: Atmospheric water vapor measurements: Comparison of microwave radiometry and lidar. J. Geophys. Res.,97, 899–916.

    • Crossref
    • Export Citation
  • Ferrare, R. A., S. H. Melfi, D. N. Whiteman, and K. D. Evans, 1992: Raman lidar measurements of Pinatubo aerosols over southeastern Kansas during November–December 1991. Geophys. Res. Lett.,19, 1599–1602.

    • Crossref
    • Export Citation
  • ———, ———, ———, ———, F. J. Schmidlin, and D. O’C. Starr, 1995: A comparison of water vapor measurements made by Raman lidar and radiosondes. J. Atmos. Oceanic Technol.,12, 1178–1195.

    • Crossref
    • Export Citation
  • Gelb, A., 1988: Applied Optimal Estimation. The MIT Press, 374 pp.

  • Han, Y., and E. R. Westwater, 1995: Remote sensing of tropospheric water vapor and cloud liquid water by integrated ground-based sensors. J. Atmos. Oceanic Technol.,12, 1050–1059.

    • Crossref
    • Export Citation
  • ———, J. B. Snider, E. R. Westwater, S. H. Melfi, and R. A. Ferrare, 1994: Observations of water vapor by ground-based microwave radiometers and Raman lidar. J. Geophys. Res.,99, 18 695–18 702.

  • Kaminski, P. G., A. E. Bryson, and S. F. Schmidt, 1971: Discrete square root filtering: A survey of current techniques. IEEE Trans. Autom. Control,AC-16, 727–736.

    • Crossref
    • Export Citation
  • Ledsham, W. H., and D. H. Staelin, 1978: An extended Kalman–Bucy filter for atmospheric temperature profile retrieval with a passive microwave sounder. J. Appl. Meteor.,17, 1023–1033.

    • Crossref
    • Export Citation
  • Martner, B. E., and Coauthors, 1993: An evaluation of wind profiler, RASS, and microwave radiometer performance. Bull Amer. Meteor. Soc.,74, 599–613.

    • Crossref
    • Export Citation
  • Melfi, S. H., and D. H. Whiteman, 1985: Observation of lower atmospheric moisture structure and its evolution using a Raman lidar. Bull. Amer. Meteor. Soc.,66, 1288–1292.

    • Crossref
    • Export Citation
  • ———, ———, and R. A. Ferrare, 1989: Observation of atmospheric fronts using Raman lidar moisture measurements. J. Appl. Meteor.,28, 789–806.

    • Crossref
    • Export Citation
  • Rodgers, C. D., 1976: Retrieval of atmospheric temperature and composition from remote measurements of thermal radiation. Rev. Geophys. Space Phys.,14, 609–624.

    • Crossref
    • Export Citation
  • Ruf, C. S., and C. T. Swift, 1988: Atmospheric profiling of water vapor density with a 20.5–23.5 GHz autocorrelation radiometer. J. Atmos. Oceanic Technol.,5, 539–546.

    • Crossref
    • Export Citation
  • Skoog, B. G., J. I. H. Askne, and G. Elgered, 1982: Experimental determination of water vapor profiles from ground-based radiometer measurements at 21.0 and 31.4 GHz. J. Appl. Meteor.,21, 394–400.

  • Stankov, B. B., B. E. Martner, and M. K. Politovich, 1995: Moisture profiling of the cloudy winter atmosphere using combined remote sensors. J. Atmos. Oceanic Technol.,12, 488–510.

    • Crossref
    • Export Citation
  • Strang, G., 1988: Linear Algebra and Its Applications. Academic, 528 pp.

  • Wang, J. R., and Coauthors, 1983: Profiling atmospheric water vapor by microwave radiometry. J. Climate Appl. Meteor.,22, 779–788.

    • Crossref
    • Export Citation
  • Westwater, E. R., 1978: The accuracy of water vapor and cloud liquid determination by dual-frequency ground-based microwave radiometry. Radio Sci.,13, 677–685.

    • Crossref
    • Export Citation
  • ———, M. T. Decker, A. Zachs, and K. S. Gage, 1983: Ground-based remote sensing of temperature profiles by a combination of microwave radiometry and radar. J. Climate Appl. Meteor.,22, 126–133.

    • Crossref
    • Export Citation
  • Whiteman, D. N., S. H. Melfi, and R. A. Ferrare, 1992: Raman lidar system for the measurement of water vapor and aerosols in earth’s atmosphere. Appl. Opt.,31, 3068–3082.

    • Crossref
    • Export Citation

i1520-0426-14-3-480-f01

Fig. 1. Comparison of transition errors computed using a 3-h radiosonde database (solid line) and a set of lidar data with a 2-min temporal resolution (dashed line).

Citation: Journal of Atmospheric and Oceanic Technology 14, 3; 10.1175/1520-0426(1997)014<0480:AOKFTD>2.0.CO;2

i1520-0426-14-3-480-f02

Fig. 2. Water vapor mixing ratio profile retrievals every half hour. Solid line—Raman lidar measurements; dashed line—retrievals; dotted line—cloud-base height. Data were collected during the FIRE II experiment.

Citation: Journal of Atmospheric and Oceanic Technology 14, 3; 10.1175/1520-0426(1997)014<0480:AOKFTD>2.0.CO;2

i1520-0426-14-3-480-f03

Fig. 3. Time series of cloud-base height identified from aerosol profiles measured by the lidar. Examples of water vapor profile retrievals during this period are shown in Fig. 2.

Citation: Journal of Atmospheric and Oceanic Technology 14, 3; 10.1175/1520-0426(1997)014<0480:AOKFTD>2.0.CO;2

i1520-0426-14-3-480-f04

Fig. 4. (a) An example of retrieval using the technique described in the text. Dash–dot line—retrieval; solid line—ground truth (from lidar); dotted line—artificial cloud base. Data were collected during FIRE II. The cloud base is created by artificially extending existing cloud base beyond clear periods. The lidar sounding is truncated at the artificial cloud base and used as cloudy measurements. The original lidar sounding is used as ground truth. (b) An example of retrieval when lidar provides no historic measurements for the portion above the cloud base. Same notation as in (a).

Citation: Journal of Atmospheric and Oceanic Technology 14, 3; 10.1175/1520-0426(1997)014<0480:AOKFTD>2.0.CO;2

i1520-0426-14-3-480-f05

Fig. 5. Time series of estimated retrieval errors. Solid line—estimated errors of the two-stage retrieval algorithm; dashed line—estimated errors of the Kalman performed at the first stage; dotted line—uncertainties of the statistical constraint.

Citation: Journal of Atmospheric and Oceanic Technology 14, 3; 10.1175/1520-0426(1997)014<0480:AOKFTD>2.0.CO;2

Save