• Banta, R. M., Newsom R. K. , Lundquist J. K. , Pichugina Y. L. , Coulter R. L. , and Mahrt L. , 2002: Nocturnal low-level jet characteristics over Kansas during CASES-99. Bound.-Layer Meteor, 105 , 221252.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Blumen, W., Banta R. M. , Burns S. P. , Fritts D. C. , Newsom R. , Poulos G. S. , and Sun J. , 2001: Turbulence statistics of a Kelvin– Helmholtz billow event observed in the nighttime boundary layer during the CASES-99 field program. Dyn. Atmos. Oceans, 34 , 189204.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chai, T., Lin C. L. , and Newsom R. K. , 2004: Retrieval of microscale flow structures from high-resolution Doppler lidar data using an adjoint model. J. Atmos. Sci, 61 , 15001520.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deardorff, J. W., 1972: Numerical investigation of neutral and unstable planetary boundary layers. J. Atmos. Sci, 29 , 91115.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deardorff, J. W., 1974: Three dimensional numerical study of the height and mean structure of a heated planetary boundary layer. Bound.-Layer Meteor, 7 , 81106.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Frehlich, R. G., and Yadlowsky M. J. , 1994: Performance of mean-frequency estimators for Doppler radar and lidar. J. Atmos. Oceanic Technol, 11 , 12171230.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Frehlich, R. G., Hannon S. M. , and Henderson S. W. , 1994: Performance of a 2-μm coherent Doppler lidar for wind measurement. J. Atmos. Oceanic Technol, 11 , 15171528.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Grund, C. J., Banta R. M. , George J. L. , Howell J. N. , Post M. J. , Richter R. A. , and Weickmann A. M. , 2001: High-resolution Doppler lidar for boundary layer and cloud research. J. Atmos. Oceanic Technol, 18 , 376393.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Harlow, F. H., and Welch J. E. , 1965: Numerical calculation of time-dependent viscous incompressible flow of fluid with free surface. Phys. Fluids, 8 , 21822189.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lenschow, D. H., Mann J. , and Kristensen L. , 1994: How long is long enough when measuring fluxes and other turbulence statistics? J. Atmos. Oceanic Technol, 11 , 661673.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lenschow, D. H., Wulfmeyer V. , and Senff C. , 2000: Measuring second- through fourth-order moments in noisy data. J. Atmos. Oceanic Technol, 17 , 13301347.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lin, C-L., Chai T. , and Sun J. , 2001: Retrieval of flow structures in a convective boundary layer using an adjoint model: Identical twin experiments. J. Atmos. Sci, 58 , 17671783.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mayor, S. D., Lenschow D. H. , Schwiesow R. L. , Mann J. , Frush C. L. , and Simon M. K. , 1997: Validation of NCAR 10.6-μm CO2 Doppler lidar radial velocity measurements and comparison with a 915-MHz profiler. J. Atmos. Oceanic Technol, 14 , 11101126.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moeng, C-H., 1984: A large-eddy-simulation model for the study of planetary boundary-layer turbulence. J. Atmos. Sci, 41 , 20522062.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moeng, C-H., and Sullivan P. P. , 1994: A comparison of shear- and buoyancy-driven planetary boundary layer flows. J. Atmos. Sci, 51 , 9991022.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Newsom, R. K., and Banta R. M. , 2003: Shear-flow instability in the stable nocturnal boundary layer as observed by Doppler lidar during CASES-99. J. Atmos. Sci, 60 , 1633.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Poulos, G. S., and Coauthors, 2002: CASES-99: A comprehensive investigation of the stable nocturnal boundary layer. Bull. Amer. Meteor. Soc, 83 , 555581.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Press, W. H., Flannery B. P. , Teukolsky S. A. , and Vetterling W. T. , 1988: Numerical Recipes in C: The Art of Scientific Computing. Cambridge University Press, 317 pp.

    • Search Google Scholar
    • Export Citation
  • Rye, B. J., and Hardesty R. M. , 1993: Discrete spectral peak estimation in Doppler lidar. I: Incoherent spectral accumulation and the Cramer-Rao bound. IEEE Trans. Geosci. Remote Sens, GE-31 , 1627.

    • Search Google Scholar
    • Export Citation
  • Rye, B. J., and Hardesty R. M. , 1997: Detection techniques for validating Doppler estimates in heterodyne lidar. Appl. Opt, 36 , 19401951.

  • Sun, J., and Crook A. , 1994: Wind and thermodynamic retrieval from single Doppler measurements of a gust front observed during Phoenix II. Mon. Wea. Rev, 122 , 10751091.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, J., and Crook A. , 1997: Dynamical and microphysical retrieval from Doppler radar observations using a cloud model and its adjoint. Part I: Model development and simulated data experiments. J. Atmos. Sci, 54 , 16421661.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, J., and Crook A. , 1998: Dynamical and microphysical retrieval from Doppler radar observations using a cloud model and its adjoint. Part II: Retrieval experiments of an observed Florida convective storm. J. Atmos. Sci, 55 , 835852.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, J., Flicker D. W. , and Lilly D. K. , 1991: Recovery of three-dimensional wind and temperature fields from simulated single-Doppler radar data. J. Atmos. Sci, 48 , 876890.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, J., and Coauthors, 2002: Intermittent turbulence associated with a density current passage in the stable boundary layer. Bound.-Layer Meteor, 105 , 199219.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Troen, I. B., and Mahrt L. , 1986: A simple model of the atmospheric boundary layer: Sensitivity to surface evaporation. Bound.-Layer Meteor, 37 , 129148.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wulfmeyer, V., Randall M. , Brewer W. A. , and Hardesty R. M. , 2000: 2-μm Doppler lidar transmitter with high frequency stability and low chirp. Opt. Lett, 25 , 12281230.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wyngaard, J. C., 1985: Structure of the planetary boundary layer and implications for its modeling. J. Climate Appl. Meteor, 24 , 11311142.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • View in gallery
    Fig. 1.

    Plot of the mean value of the percent deviation of R from unity, 100|R − 1|, as a function of the scaling parameter a. The quantity R is given by Eq. (24) and 100|R − 1| was averaged over 10 realizations of the random vector δ

  • View in gallery
    Fig. 2.

    Flow diagram of the retrieval algorithm

  • View in gallery
    Fig. 3.

    Map of the CASES-99 main field site showing the scan coverage (solid line), the model domain boundaries (dotted line), the lidar (HRDL), and the 60-m tower. Within the scan coverage area the terrain elevation varies by about 80 ft (25 m). The lowest point occurs in the gully near the southernmost portion of the scan. The highest point occurs on a ridge just to the north of this gully

  • View in gallery
    Fig. 4.

    Samples of radial velocity fields used in the assimilation trials. These images show quality controlled radial velocity at an elevation angle of 2° from (a) the first and (b) the second volume scans

  • View in gallery
    Fig. 5.

    Base-state wind and virtual potential temperature profiles. Base-state virtual potential temperature profile, θb, was obtained from a radiosonde released at about 1900 UTC (1400 CDT) from the main field site. Base-state wind profiles (ub, υb) were computed from the lidar volume scan data using a VAD-type processing technique

  • View in gallery
    Fig. 6.

    Range–time displays of (a) radial velocity and (b) the logarithm of wSNR. These data were recorded on 19 Oct 1999 as the lidar stared nearly horizontally toward the north. The winds were southerly and relatively strong. The tilted linear features apparent in the radial velocity data were created by gust structures propagating downwind with time

  • View in gallery
    Fig. 7.

    (a) Individual time series of radial velocity from the data shown in Fig. 6a. Solid line (dotted line) is a time series at a range of 600 m (2400 m) from the lidar. (b) Autocovariance functions (ACFs) of the time series shown in (a)

  • View in gallery
    Fig. 8.

    Estimated radial velocity measurement precision as a function of wSNR

  • View in gallery
    Fig. 9.

    Measurement precision fields corresponding to the radial velocity data shown in Fig. 4

  • View in gallery
    Fig. 10.

    (a) Plot of Jobs normalized by its starting value as a function of iteration for retrieval 49. (b) Distribution of Δm/σm for retrieval 49. The distribution in (b) indicates that 87% of the model generated radial velocities occur within the estimated error of the measured radial velocities

  • View in gallery
    Fig. 11.

    Horizontal cross sections at z = 100 m for retrieval 49. (a) Perturbation horizontal velocity vector field; regions of horizontal convergence (negative divergence) are shaded. (b)–(d) Perturbation virtual potential temperature, vertical velocity, and vertical vorticity, respectively; shaded areas indicate negative values. Regions labeled by A, B, and C are discussed in the text

  • View in gallery
    Fig. 12.

    Vertical cross sections at y = 0 m for retrieval 49. (a) Perturbation u and w vector field. (b) Perturbation virtual potential temperature. Shaded areas indicate negative values. Regions labeled by A and C are discussed in the text and correspond to structures observed in the horizontal cross sections shown in Fig. 11

  • View in gallery
    Fig. 13.

    Comparisons between retrievals 38 and 43. (a), (c) Horizontal cross sections of the perturbation horizontal velocity vector fields for retrieval 38 and 43, respectively; regions of negative vertical velocities are shaded gray. (b), (d) Horizontal cross sections of the perturbation virtual potential temperature for retrieval 38 and 43, respectively; shaded regions indicate negative perturbation virtual potential temperature

  • View in gallery
    Fig. 14.

    Rms deviations and linear correlation coefficients between retrievals 38 and 43 as functions of range from the lidar. The solid curve in (a) shows the rms deviation between u from retrieval 38 and u from retrieval 43. Rms deviations for υ, w, and θ are shown with the dotted, dashed, and dash–dotted curves, respectively. The solid curve in (b) shows the median linear correlation coefficient between u from retrieval 38 and u from retrieval 43. Median linear correlation coefficients for υ, w, and θ are shown with the dotted, dashed, and dash–dotted curves, respectively

  • View in gallery
    Fig. 15.

    Vertical profiles of horizontally averaged variances for (a) u, (b) υ, (c) w, and (d) θ. Solid curves are the results for retrieval 38 and the dotted curves are the results for retrieval 43

  • View in gallery
    Fig. 16.

    Vertical profiles of horizontally averaged kinematic heat flux for retrieval 38 (solid line) and retrieval 43 (dotted line)

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 167 30 7
PDF Downloads 109 90 3

Assimilating Coherent Doppler Lidar Measurements into a Model of the Atmospheric Boundary Layer. Part I: Algorithm Development and Sensitivity to Measurement Error

Rob K. NewsomCooperative Institute for Research in the Atmosphere, Colorado State University, Fort Collins, Colorado

Search for other papers by Rob K. Newsom in
Current site
Google Scholar
PubMed
Close
and
Robert M. BantaNOAA/Environmental Technology Laboratory, Boulder, Colorado

Search for other papers by Robert M. Banta in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

A four-dimensional variational data assimilation (4DVAR) algorithm for retrieval of spatially and temporally resolved velocity and thermodynamic fields within the atmospheric boundary layer (ABL) is described and applied to a coherent Doppler lidar dataset. The adjoint method is used to find the initialization of an ABL model that gives the best fit to radial velocity measurements from the Doppler lidar. The adjoint equations are derived by assuming that subgrid-scale fluxes can be represented as general functions of the resolved-scale rates of strain and potential temperature gradients. For this study, particular attention is paid to the treatment of real measurement error. Radial velocity precision as a function of the signal-to-noise ratio (SNR) is estimated from time series analysis of real fixed beam data, and this information is used in the evaluation of the cost function. The cost function is evaluated by interpolating the model output to the observation coordinates. As a result, the error covariance matrix retains its diagonal structure and the form of the cost function is simplified.

The retrieval method is applied to Doppler lidar data collected under convective conditions during the Cooperative Atmosphere/Surface Exchange Study (CASES-99) field program. The impact of the SNR-dependent measurement error is investigated by comparing a retrieval using equally weighted data to a retrieval using the estimated velocity precisions. At near range the fields are well correlated. However, at longer range, as the velocity precision exceeds the standard deviation of the measurements, the correlation decreases rapidly. Furthermore, retrievals using equally weighted data produce higher variances.

Corresponding author address: Rob K. Newsom, Harris Corporation, P.O. Box 37, MS: W3-3330, Melbourne, FL 32902. Email: rnewsom@harris.com

Abstract

A four-dimensional variational data assimilation (4DVAR) algorithm for retrieval of spatially and temporally resolved velocity and thermodynamic fields within the atmospheric boundary layer (ABL) is described and applied to a coherent Doppler lidar dataset. The adjoint method is used to find the initialization of an ABL model that gives the best fit to radial velocity measurements from the Doppler lidar. The adjoint equations are derived by assuming that subgrid-scale fluxes can be represented as general functions of the resolved-scale rates of strain and potential temperature gradients. For this study, particular attention is paid to the treatment of real measurement error. Radial velocity precision as a function of the signal-to-noise ratio (SNR) is estimated from time series analysis of real fixed beam data, and this information is used in the evaluation of the cost function. The cost function is evaluated by interpolating the model output to the observation coordinates. As a result, the error covariance matrix retains its diagonal structure and the form of the cost function is simplified.

The retrieval method is applied to Doppler lidar data collected under convective conditions during the Cooperative Atmosphere/Surface Exchange Study (CASES-99) field program. The impact of the SNR-dependent measurement error is investigated by comparing a retrieval using equally weighted data to a retrieval using the estimated velocity precisions. At near range the fields are well correlated. However, at longer range, as the velocity precision exceeds the standard deviation of the measurements, the correlation decreases rapidly. Furthermore, retrievals using equally weighted data produce higher variances.

Corresponding author address: Rob K. Newsom, Harris Corporation, P.O. Box 37, MS: W3-3330, Melbourne, FL 32902. Email: rnewsom@harris.com

1. Introduction

Progress in understanding and modeling the atmospheric boundary layer (ABL) is important for improvement in the ability to address many important applications, including air quality, emergency response and homeland security, diffusion modeling, and the representation of ABL processes in numerical weather prediction (NWP) models.

Because of the turbulent nature of the ABL, processes in this layer have been studied statistically. Eulerian statistical quantities have been measured using time series from towers and spatial series from instrumented aircraft for periods when the atmosphere could be argued to be stationary. However, Wyngaard (1985) and Lenschow et al. (1994) have pointed out that, for many important quantities, including higher-order statistics and cross correlations (covariances), a statistically stable estimate would require a time series considerably too long for these techniques. Wyngaard (1985) argued that a feasible way to achieve the required sample size would be through obtaining area or volume data, as from a remote sensing system. A scanning Doppler lidar system capable of providing wind velocity measurements over an area (sector) or volume of the boundary layer has been developed (Grund et al. 2001) and used to study features of the nocturnal boundary layer (Blumen et al. 2001; Newsom and Banta 2003; Poulos et al. 2002; Sun et al. 2002). This instrument is called the high-resolution Doppler lidar (HRDL). Individual azimuth or elevation scans, at spatial resolutions of a few tens of meters along the lidar beam and 10 m or less in the scanning direction, sweep out 2D areas within the ABL. By repeating these scans while incrementing the angle orthogonal to the scan plane HRDL can be used to sweep out 3D volumes of data.

Techniques to retrieve all three wind components and the thermodynamic fields have been developed for scanning Doppler radar. These techniques require using repeated 3D volume scans. In the present study we adapt this technique to Doppler lidar, which provides finer resolution (and higher precision) data over a smaller sampling volume than radar. The retrieval technique provides (u, υ, w, θ, p) over a volume of the ABL that may be several kilometers across at data intervals that resolve the large eddies of the ABL, at least for the daytime unstable mixed layer or convective boundary layer (CBL). In this regard, the retrieval technique provides a field that is similar to large-eddy-simulation (LES) output except that the scale and magnitude of the fluctuations are constrained, based on measured quantities.

LES has proven an important tool in the study of the ABL (e.g., Deardorff 1972, 1974; Moeng 1984; Moeng and Sullivan 1994). Typically profiles are calculated as the vertical variation of horizontal mean and mean fluctuating quantities (variances, covariances, higher-order moments, etc.), after the model has achieved stationarity. Important issues are how to verify the LES results and how the results can be applied to understanding adjustment processes in nonstationary boundary layers. The volume retrieval techniques just described, which are the subject of this paper, offer hope that these verification and interpretation issues can be directly addressed by the retrieved analysis of the ABL variables.

Whether these techniques can be useful depends on a number of factors, including those related to instrument capabilities and those related to the numerical retrieval scheme. Key instrumental requirements include the spatial resolution, range, sampling frequency, volume repeat frequency, and precision of the measurements needed for sufficiently accurate retrieval. These are competing requirements necessitating trade-off; for example, the repeat frequency of the scan volume can be increased by scanning faster, but the faster scanning degrades the spatial resolution (in the scan direction). The compromises are more favorable for instruments that operate at higher signal-to-noise ratio (SNR) and higher sampling frequencies, so important questions are: does current technology permit useful fields to be retrieved and what will be the impact of improvements expected in the future?

Numerical questions include, what is the required sophistication of the forward model, can an adjoint be formulated as the model becomes more complex, what is the required model resolution for accurate retrievals, what is the sensitivity of the results to input quantities— such as boundary condition (BC) treatment and first-guess fields, and will the resulting measurement–modeling systems produce accurate enough representations of the eddy fields to produce useful analysis of statistical quantities?

Here, we describe a retrieval model that will be used to explore some of these questions. A dynamically consistent approach to the retrieval problem involves fitting the output of a prognostic model to the lidar measurements. If the model's BCs are prescribed in some manner, then the solution will be uniquely determined by the initial conditions. Thus, the initial conditions can be regarded as control parameters that are adjusted to optimize the agreement between the lidar observations and the model prediction of radial velocity. The retrieved fields are obtained when the optimal initial state is determined. This procedure, which is referred to as four-dimensional variational data assimilation (4DVAR), forms the basis of the retrieval technique employed in this study.

This method was initially demonstrated by Sun et al. (1991) using simulated data and later applied to a dry gust front case using real Doppler radar observations (Sun and Crook 1994). More recently, Sun and Crook (1997, 1998) have adapted and applied the technique to study the structure and dynamics of convective storms. Lin et al. (2001) conducted a series of identical twin experiments using a modified version of Sun's original algorithm with synthetic data generated by LES to explore the potential of retrieving microscale flow structures from single-Doppler lidar data. Modifications included the introduction of a surface layer parameterization scheme based on Monin–Obukhov similarity theory. Chai et al. (2004) then incorporated a variable eddy diffusion profile and applied this algorithm to HRDL data collected under clear-sky, daytime convective conditions during the Cooperative Atmosphere/Surface Exchange Study (CASES-99) field campaign.

The present study also focuses on the retrieval of microscale flow structures from Doppler lidar data using 4DVAR. The method presented here differs in several respects from the method described by Lin et al (2001) and Chai et al. (2004). Some of the more significant differences include the following.

  1. Data ingest scheme: Quality-controlled lidar velocity data are input directly, and the model fields are interpolated to the observation coordinates so that the error covariance matrix assumes a diagonal structure.

  2. Treatment of measurement errors: Fixed-beam lidar data are used to estimate the relationship between radial velocity measurement error and the average SNR. This information is then used to properly weight measurements in the cost function.

  3. Buoyancy forcing: This is expressed in terms of a virtual potential temperature θ perturbation from the horizontal mean. This formulation is consistent with the standard treatment in LES studies of the ABL.

  4. Generic eddy diffusivity formulations: In future studies we wish to use the 4DVAR retrieval algorithm as a means of evaluating various candidate subresolution-scale (SRS) turbulence parameterization schemes. In anticipation of this, we have included a derivation of the adjoint equations with a generalized representation of the SRS fluxes.

The second point listed above addresses the treatment of measurement errors. As far as we are aware, the impact of the strong range dependence of the SNR on retrievals obtained using 4DVAR has not been considered in previous studies. The SNR, which is the ratio of the coherent signal energy to the average spectral noise level, provides one performance measure for a coherent Doppler lidar. As the SNR decreases due to two-way atmospheric extinction and 1/r2 attenuation of the backscattered energy, the quality of radial velocity measurements degrades. In order to weight the measurements properly in the formulation of the cost function it is necessary to relate SNR to measurement error. For this study a relationship between measurement error and SNR is obtained experimentally assuming that the dominant source of error is random noise induced by the mean-frequency estimator. The standard deviation of the random measurement error is referred to as the velocity precision of the lidar, σ.

In this paper we describe a 4DVAR scheme and use it with Doppler lidar data taken during the CASES-99 field program to address many of these questions. This paper describes the present technique, presents results using data collected under clear-sky convective conditions during CASES-99, and examines the impact of the SNR dependent measurement error by comparing a retrieval using equally weighted data with a retrieval using estimated velocity precisions. In Part II we explore the sensitivity of retrieved wind and temperature fields to prescribed base-state potential temperature profiles, eddy viscosity profiles, model resolution, and the length of the assimilation period. Also in Part II we investigate the characteristics of statistical profiles derived from the retrieved fields.

This paper is organized as follows. The retrieval method is described in section 2. In section 3 Doppler lidar observations are presented. This section also describes a method for estimating the radial velocity precision as a function of signal-to-noise ratio. The retrieval algorithm is applied in section 4. Retrieval experiments are conducted to evaluate the effect of neglecting the SNR-dependent measurement precision. Our conclusions are presented in section 5.

2. Retrieval method

This section describes the various components of the wind and temperature retrieval algorithm. The forward model consists of a set of equations describing the motion of the flow. A quantitative measure of the “distance” or discrepancy between the observations and the model output is the cost function, described in section 2b. The forward model used here is similar to that used in the studies by Sun and Crook (1994), Lin et al. (2001), and Chai et al. (2004). The principal differences in the current approach were described in the previous section.

The initial conditions of the forward model are adjusted in order to minimize the cost function. Minimizing the cost function is an iterative process requiring information about how the cost function changes with respect to changes in the initial state of the forward model. This information is provided by the solution of the so-called adjoint model, which is derived from the forward model using variational principles. Later in this section we describe the mechanics of the retrieval algorithm and the implementation of a conjugate gradient method for minimizing the cost function.

a. Forward model

The retrieval algorithm uses a forward model that simulates incompressible dry, shallow flow using the Boussinesq approximation. The forward model equations are given by
i1520-0426-21-9-1328-e1
Summation notation is assumed for any i, j, or k index repeated in a term. The acceleration of gravity is g, Θref is a reference virtual potential temperature, p is the pressure normalized by a constant air density, fc is the Coriolis parameter, and ɛijk is the Levi–Civita symbol. The geostrophic wind ug = (ug, υg, 0) is assumed to depend only on height z. Horizontally averaged variables are enclosed in angle brackets. The virtual potential temperature is θ = θ̂ + θb, where θb is the a prescribed base-state virtual potential temperature and θ̂ is the departure from the base state. The base-state potential temperature θb(z) is prescribed from radiosonde observations and is a function of height only.
Subresolution-scale (SRS) fluxes of momentum and heat are represented by ϕij and ψi, respectively. In order to maintain a degree of generality we assume that the SRS fluxes depend explicitly on the local resolved-scale velocity gradient
i1520-0426-21-9-1328-e5
potential temperature gradient
gθ,
and virtual potential temperature θ. Thus, ϕij = ϕij(s, g, θ) and ψi = ψi(s, g, θ). The adjoint equations are initially developed using this relatively general SRS model. Later in section 3d, a specific SRS parameterization scheme will be considered.

Equation (4) is a diagnostic equation for the pressure that is derived from the divergence of Eq. (1) and is used to maintain incompressibility. The Harlow–Welch (1965) scheme is used in Eq. (4) to suppress divergence, and Δt is the integration time step size. The forward model and its adjoint are integrated using a space-centered, forward-in-time scheme on a staggered Cartesian grid with periodic lateral BC. At the top and bottom of the domain the vertical velocity w vanishes and θ is set to its base-state value. Horizontal velocities are set to zero at the bottom of the domain. At the top of the domain the horizontal velocities are prescribed from observations derived from a VAD-type (Banta et al. 2002; Chai et al. 2004) analysis of the lidar data.

b. Cost function and interpolation procedure

The cost function, a quantitative measure of the difference between the measurements and the model output, in this study is given by
JJobsJd
The primary constraint Jobs is the error or difference between the measured radial velocities from the lidar and the radial component of the velocity field generated by the forward model. A second constraint or penalty term Jd is included to suppress divergence in the initial velocity field.
In evaluating Jobs the calculation is greatly simplified if the error covariance matrix is diagonal. In this case Jobs is given by
i1520-0426-21-9-1328-e8
where
mumrmuobsrm
The observed radial velocity is uobsrm, σm is the corresponding measurement error, and rm is the unit vector from the lidar to the mth observation. The velocity field generated by the forward model is denoted by um, where the overbar implies interpolation to the coordinates of the mth observation. The summation in (8) is performed over all radial velocity observations within the model domain and analysis time period.

As discussed in section 3, individual radial-velocity estimates are independent and the measurement errors, random and uncorrelated. If we also assume the errors are unbiased, then the error covariance matrix is diagonal and Jobs takes the simple form in (8).

The approach often applied to radar data analysis is to interpolate the measurement data to the model grid (Sun and Crook 1994, 1998). This approach can be problematic for lidar data due to disparities in spatial and temporal resolution. Lidars can spatially resolve turbulent eddies in the unstable CBL. However, the observations may not be well resolved in time for these spatial scales because volume scans can take one to a few minutes to complete. In that time the structure of the small eddies can change. Thus, in interpolating the observations to each model grid point, a substantial degree of spatial and temporal smoothing may result. The smoothing may be so significant as to filter the small eddy structures out of the measurement data as used in the retrieval. Another consequence of interpolating measured data to the model grid is in the formal treatment of measurement errors in Jobs. Interpolating the measured data to the model grid causes correlations in the measurement errors, producing off-diagonal terms in the error covariance matrix so that the error covariance matrix corresponding to interpolated data is generally not diagonal. In that case, the formal treatment of the measurement error is complicated. The approach adopted here involves interpolating the model fields to the measurement coordinates, rather than interpolating the measured values to the model grid. The error covariance matrix then retains its diagonal structure. Thus, in addition to avoiding a procedure that can unduly smooth out small-scale structure in the data, the formulation of the cost function is greatly simplified by this approach.

The interpolation is essentially a weighted average of u that can be expressed as
i1520-0426-21-9-1328-e10
where Σr implies summation over the model domain nodes, tm is the time of the mth observation, and tn is the time of the nth time step of the model output. For each observation m, the weighting function W is normalized such that ΣN−1n=0 Σr W(rrm, tntm) = 1. This could introduce a significant computational burden if all the W had to be computed each time because the retrieval algorithm generally requires many cost function evaluations. To alleviate these extra calculations, the averaging weights for each measurement are calculated in advance and then stored for future use during the retrieval run.

We use linear interpolation in space and time to interpolate the model variables to the observation coordinates. For linear interpolation, only the nearest model grid points contribute to the interpolated value. A given observation is enclosed within a space–time grid cell that is defined by 16 vertices. Interpolating a model variable to the observation coordinate involves a weighted average of those 16 vertices. Thus, to evaluate (10) efficiently it is only necessary to precalculate and store 16 weighting parameters as just described and the indices of the enclosing grid cell for each observation. This can easily be accomplished using less than 10 Mb of storage for the applications considered here.

In addition to Jobs it is necessary to include the penalty term Jd in the cost function to suppress the divergence of the velocity field at the initial time step (Sun et al. 1991). The divergence penalty term is given by
i1520-0426-21-9-1328-e11
where D0 = ∇ · u0. The dimensional weighting parameter kd is used to control the contribution of Jd to the cost function. As a general rule, Jobs should dominate the cost function. For this study, kd is set such that Jd contributes less than 10% of the cost function.

c. The adjoint equations

The adjoint model provides an efficient means of computing the gradient of the cost function with respect to the initial state of the forward model. A straightforward way to derive the adjoint equations is to convert the constrained minimization problem into an unconstrained problem through the use of the Lagrange function. The Lagrangian, L, is constructed by appending the model equations to the cost function. The minimum of J is found by determining the stationary point of L with respect to the model's prognostic and diagnostic variables.

For consistency, the adjoint equations should be derived from the discrete representation of the forward model. The current model uses a space-centered, forward-in-time integration scheme. The Lagrangian is defined to be
i1520-0426-21-9-1328-e12
where ũ, θ̃, and are the Lagrange multipliers (adjoint variables) corresponding to u, θ, and p. The time-step index is indicated as a superscript on any variable, and Δt is the time-step size. The adjoint variables are assumed to be horizontally periodic for BC specification, consistent with the model variables. In (12) we have defined
i1520-0426-21-9-1328-e13
We require that the first variation of L with respect to all variables vanish for t > 0. For this study, we follow the general variational procedure described by Sun et al. (1991). The details of that calculation for this particular model are given in appendices A, B, and C. The essential result is that
i1520-0426-21-9-1328-e16
Thus, the gradient of L with respect to the initial model state is given by the negative of corresponding adjoint variable at the initial time. To find ũ0 and θ̃0 one must first integrate the forward model forward in time and then integrate the adjoint equations backward in time, as described in appendix A.

d. SRS fluxes

Appendix A presents expressions for the adjoint equations assuming that the SRS fluxes are given as generic functions of the resolved-scale velocity gradient, θ gradient, and θ. In order to apply the retrieval method to real lidar data, we consider specific expressions for the SRS fluxes of heat and momentum using standard gradient transport theory. The relationship between SRS fluctuations and resolved-scale gradients in u and θ are assumed to obey
i1520-0426-21-9-1328-e18
where is the SRS turbulent kinetic energy, sij = ∂ui/∂xj, and gi = ∂θ/∂xi. The coefficients of eddy diffusion and diffusivity are Km and Kh, respectively. If we let
i1520-0426-21-9-1328-e20
then the forward model remains unchanged, provided we redefine the pressure such that pp + 2e/3.
For this study, Km and Kh are prescribed functions of height only. The vertical variation of Km is modeled using an expression similar to the one proposed by Troen and Mahrt (1986). This is given by
i1520-0426-21-9-1328-e22
where zmax is the height of the computational domain, Kmax determines the maximum value of Km, and α controls the shape of the profile and the height of the maximum. We further assume that
KhKm
Appendix C presents the adjoint equations using this specific formulation of the SRS fluxes.

e. Adjoint validation

The adjoint code can be tested by comparing the gradient of L as determined by the adjoint method to the gradient determined using a finite difference approximation (Sun et al. 1991). This test is performed by defining
i1520-0426-21-9-1328-e24
where x is a vector containing the initial u and θ fields, and δx is a small departure from x. The difference L(x + δx) − L(x) is computed by direct evaluation of the cost function, whereas ∂L/∂x is computed from Eqs. (16) and (17). For R = 1 Eq. (24) is essentially a first-order Taylor series expansion of L. If the adjoint equations have been properly formulated and implemented, then R ≈ 1 for small δx. To evaluate (24) we let δx = , where a is a scaling parameter and δ is a vector containing random values of u and θ ranging between ±0.5 m s−1 and ±0.5 K, respectively. For a given realization of δ Eq. (24) is evaluated for scaling parameter a ranging from 1.0 to 10−7. The percent deviation of R from unity is given by 100|R − 1|. Figure 1 shows 100|R − 1| versus a averaged over 10 realizations of δ. To obtain this result we used the lidar data described in section 3. The vector x was set to the base-state profiles of u and θ, and the computational domain was set up as described in section 4 using a grid resolution of 24 × 24 × 20. This test shows that, on average, R converges to within 0.6% of unity.

f. Optimization procedure

Figure 2 shows a flow diagram describing the wind and temperature retrieval method. A conjugate gradient method based on the Polak–Ribiere algorithm (Press et al. 1988) is used to minimize the cost function. The process is initiated by assigning a “first guess” to the wind and potential temperature fields. For this study, the first-guess fields are set to the base-state wind profile obtained from a VAD-type analysis of the lidar data, and the base-state temperature profile from radiosonde data.

The first step inside the main loop is to evaluate the cost function. Each cost function evaluation involves integrating the forward model from t = 0 to t = (N − 1)Δt. The output of the forward model is then used to integrate the adjoint equations backward in time from t = (N − 1)Δt to t = 0. The gradients provided by Eqs. (A3), (16), and (17) are used to establish a (conjugate) search direction. The algorithm then performs a “line minimization” (Press et al. 1988) by evaluating the cost function along the search direction. This step generally involves several cost function evaluations. Once the line minimum has been found, the initial conditions of the forward model are updated and the whole process is repeated until the cost function ceases to change within a predefined tolerance. We note that one iteration or cycle in the main loop involves a single integration of the adjoint equations and multiple integrations of the forward model.

3. Observations and measurement precision

The retrieval algorithm is applied to data from NOAA's high-resolution Doppler lidar taken during the Cooperative Atmosphere/Surface Exchange Study in October of 1999 (Poulos et al. 2002). The CASES-99 field site was located in south-central Kansas in an area characterized by open grassland and very gently rolling terrain. Figure 3 shows a map of the field site and indicates the locations of selected instruments.

HRDL is a scanning, coherent, solid-state Doppler lidar that was specifically designed for atmospheric boundary layer research (Grund et al. 2001). It was developed at NOAA's Environmental Technology Laboratory (NOAA/ETL) in cooperation with the U.S. Army Research Office (ARO) and NCAR's Atmospheric Technology Division (NCAR/ATD). HRDL operates in the near-infrared (2.02 μm), and is sensitive to scattering from aerosol particles. Basic performance characteristics for HRDL are summarized in Table 1. Grund et al. (2001) and Wulfmeyer et al. (2000) discuss the design and performance of this lidar in greater detail.

a. Observations

HRDL recorded a total of eight volume scans between 2055:20 and 2107:21 UTC during the afternoon of 25 October 1999. Volume data were taken using a stepped sector-scan technique. The laser beam was scanned between 240° and 300° in azimuth at a rate of 12° s−1. The elevation angle was stepped in 2° increments between 0° and 20°. Lidar data were reprocessed with 60-m range gates and 100-pulse averages, resulting in a beam rate of 2 Hz. The beam rate is the pulse repetition frequency multiplied by the number of pulses averaged. The reprocessing of the data was from raw data recorded in the field. The volume-scan repeat rate was approximately 1.5 min. The mean winds were light so that animations of the radial velocity field show good correlation in the structure from scan to scan. For this study we use two consecutive volume scans recorded during a 3-min period from 2059:51 to 2102:50 UTC (1559: 51 to 16:02:50 CDT).

Basic quality control involved removing radial velocity measurements corresponding to hard target returns and low SNR. As discussed below, removal of low SNR data is effective at eliminating not only noisy data, but also data that may be contaminated by systematic error. Data were also excluded for ranges within HRDL's dead zone (<280 m) and greater than 2700 m. Figure 4 displays quality controlled radial velocity data from the two volume scans used in this study. These images show radial velocity structures typical of the unstable CBL.

In addition to the radial velocity field the retrieval algorithm also requires specification of a base-state θ profile. This information was obtained from radiosonde data using NCAR's Global Positioning Atmospheric Sounding System (GLASS). This study uses data from a radiosonde released at approximately 1900 UTC (1400 CDT) or about 2 h prior to the acquisition of the volume data. Figure 5 shows the base-state θ profile obtained from that sounding. Also shown is the base-state wind profile. The wind profile was computed from the lidar volume scan data using a VAD-type processing technique described in Banta et al. (2002) and Chai et al. (2004).

During the time of the volume scans the sky was clear and the boundary layer was moderately unstable. The profiles in Fig. 5 indicate a shallow CBL with light westerly winds (∼2 m s−1). The base of the capping inversion was located at approximately 600 m AGL. Above this level the winds veered with height and the wind speeds increased. At 800 m AGL the winds were northwesterly at about 5 m s−1. Table 2 shows estimates of velocity variances, friction velocity, and the kinematic heat flux obtained from sonic anemometers on the main 60-m tower (see Fig. 3). Eight sonic anemometers were mounted at heights of 1.5, 5, 10, 20, 30, 40, 50, and 55 m AGL.

b. Velocity precision

In general, radial velocity measurements contain both systematic (bias) and random errors. Possible causes of systematic error include frequency drifts in the lasers, nonlinear amplifiers, or digitization errors. Random errors are determined by the signal statistics and the performance of the mean-frequency estimator (Frehlich et al. 1994).

For strong signals the noise in the radial-velocity measurement is very nearly normally distributed and strongly peaked. In the low-signal regime Doppler spectra are dominated by spurious peaks that are randomly distributed across the receiver passband of ±25 m s−1 (Rye and Hardesty 1993, 1997). The finite passband tends to bias the mean radial velocity toward zero in the low signal regime.

Simulations indicate that radial velocity errors are essentially unbiased for large SNR (Rye and Hardesty 1993; Frehlich and Yadlowsky 1994). Frehlich et al. (1994) evaluated the performance of a 2-μm lidar that was similar in many respects to HRDL and found from the analysis of hard target returns (high SNR) that the dominant source of error was random noise induced by the mean-frequency estimator. Analysis of hard target fixed-beam data taken with HRDL during CASES-99 also did not indicate the presence of a significant bias. Accordingly, we assume that errors in the data used in this study are dominated by random noise in the strong signal regime.

The random noise component of the radial velocity signal is estimated from a time series analysis of fixed-beam data by assuming that the time scale for variations in the true radial velocity is much longer than the time scale associated with noise fluctuations. In the present case, this is a good assumption since the beam rate is on the order of 2 Hz. The variance of the measurement noise was estimated by analyzing the fixed-beam data shown in Fig 6. These data were taken as the beam was held fixed in a horizontal orientation, parallel to the mean winds, at an azimuth of 352.8° and an elevation of 1.88°. The data were processed using 60-m gates and 100-pulse averages.

For a given range gate, the noise variance is estimated from the autocovariance function (ACF) of the radial-velocity time series. Random noise in the signal results in a delta function spike at zero time lag in the ACF (Mayor et al. 1997; Lenschow et al. 2000). An estimate of the noise variance is obtained by taking the difference of the ACF values at the first and zeroth time lags. The velocity precision is then given by the square root of this difference.

Figure 7a shows time series of radial velocity at two range gates from Fig 6. The radial velocity signal at a range of 600 m (solid line) corresponds to a relatively strong return signal, whereas the radial velocity signal at a range of 2400 m (dotted line) corresponds to a much weaker return signal. Autocovariance functions of these two time series are shown in Fig 7b. The ACF of the signal at 600 m indicates very little noise relative to the atmospheric contribution. By contrast, the ACF of the signal at 2400 m clearly shows a pronounced noise spike in the zeroth time lag.

The analysis described above can be used to determine velocity precision as a function of range. However, it is more useful to relate velocity precision to average signal strength since the range performance of the lidar can change from day to day depending on the concentration and characteristics of particulates in the atmosphere. In this case, the wideband signal-to-noise ratio (wSNR) is used as a measure of the signal strength. The wSNR is defined as the signal energy integrated over the passband to the average noise energy integrated over the passband (Rye and Hardesty 1993). A plot of velocity precision versus the time-mean wSNR is displayed in Fig 8. This curve shows that according to this method of analysis, HRDL's velocity precision approaches 15 cm s−1 at high wSNR. In the very low wSNR regime the precision approaches an asymptotic limit as radial velocities become uniformly randomly distributed across the finite passband.

The curve shown in Fig 8 is used to assign measurement errors to individual radial velocities based on the corresponding wSNR measurements. Figure 9 shows the velocity precision field corresponding to the radial velocity data displayed in Fig. 4. The measurement error varies smoothly and is generally less than 0.5 m s−1 for ranges less than roughly 1800 m. Beyond 1800 m the variation in the velocity precision field is more random and contains numerous spikes exceeding 1.0 m s−1. We note that for this study the radial velocities corresponding to wSNR values less than ∼0.04 were not used in the assimilation trials. For the remaining data, random errors were approximately less than or equal to 2 m s−1.

4. Retrievals

Retrievals were performed using a computational domain with an origin centered on the lidar. As indicated in Fig 3, the domain extended from −3 to 0 km in x, and from −1.5 to 1.5 km in y. In the vertical dimension the domain extended from 0 to 800 m. The scan volume is contained well within the boundaries of the domain. This provides a buffer zone, which helps to minimize effects induced by the artificial lateral boundary conditions (Chai et al. 2004).

All trials reported here use α = 4 for the eddy diffusion profiles. Tests were conducted prior to running the retrieval algorithm to determine an appropriate value of Kmax for each grid resolution. These tests were performed by allowing the forward model to spin up using a random initial perturbation temperature field. Values of Kmax used in the retrievals were selected that resulted in approximately steady values for the volume-averaged resolved-scale TKE from the tests.

For all retrievals the algorithm was allowed to continue for 200 iterations before being terminated. In all, the retrievals performed here use 11 274 individual radial-velocity measurements in the volume scans used. We used relatively modest computational resources to perform the retrievals. All retrieval runs were performed on a PC with 1.5 Gb of internal memory and a single 1.5-GHz Pentium 4 processor.

Table 3 provides a description of the retrievals performed. Trials 38 and 43 used a grid of 24 × 24 × 20 points, which results in slightly less than 45 000 control parameters. The retrievals are computationally intensive, and the analyses of the resulting data also require considerable computational resources. Use of this grid was adequate to probe differences between runs for the purpose of the sensitivity studies performed here and in Part II.

Trial 49 was run at higher resolution to test the effects of resolution of the computational grid on the results. This run used a grid of 40 × 40 × 34, which results in a little more than 214 000 control parameters. It will be shown in Part II that this increased resolution had a significant impact on the magnitude of the calculated fluctuations and fluxes. Here we use this run to illustrate the capabilities of the retrieval algorithm to produce plausible perturbation fields of nonmeasured quantities, for example, w and θ.

Application of the retrieval algorithm is first demonstrated using the high-resolution grid. We then examine the effects of wSNR-dependent velocity precision by comparing the results of two retrievals, one with equally weighted data (trial 43) and one with data weighted by the estimated velocity precisions (trial 38).

a. High-resolution retrieval

As shown in Table 3, the high-resolution run (trial 49) used Kmax = 7 m2 s−1 and Δt = 1 s. Figure 10a shows the value of Jobs normalized by its initial value as a function of the iteration number. For this retrieval the cost function converged to about 0.16 of its starting value after about 50 iterations. A measure of how well the algorithm fits the observed radial-velocity field is provided by a histogram of the difference between measured radial velocities and those calculated from the retrieved wind field. Figure 10b shows such a histogram, accumulated over the entire assimilation period. For this retrieval, 87% of the model generated radial velocities, u · r, occur within ±σm of the measured radial velocities uobsrm. We also found no significant change in the distribution as a function of time.

Figures 11 and 12 show horizontal and vertical cross sections of the retrieved fields at the midpoint of the assimilation time. The large-eddy structure shows the relations among regions of convergence or divergence at 100 m AGL, regions of warm or cold deviations, and regions of upward or downward motions. For example, region A is a region of relatively cool sinking motion associated with low-level divergence, whereas B and C are warm, rising regions with near-surface convergence. Figure 12 shows that A and C extend vertically through the CBL. These cross sections show that, even though these regions show overall updraft or downdraft characteristics, considerable fine structure exists within the regions.

b. Effects of measurement error

Trials 38 and 43 were performed to evaluate the effect of the SNR-dependent measurement errors. As shown in Table 3, both of these retrievals used Kmax = 10 m2 s−1 and Δt = 2 s. Trial 38 used the SNR-dependent measurement errors, whereas for trial 43 all measurements were equally weighted. In that case, the contributions to the cost function from regions of low wSNR were given as much weight as regions of high wSNR. For trial 38 contributions to the cost function from regions of low wSNR were suppressed. Figure 13 shows a comparison of horizontal cross sections between trials 38 and 43. Overall the gross features are similar, although the fields become noisier in trial 43 at greater range from the lidar (lower SNR). We note here that the discrepancies produced by this effect are not as strong as those due to other effects that will be described in Part II.

Figure 14 shows the rms deviation and median temporal correlation between trials 38 and 43 as a function of range from the lidar, thus quantifying the effects illustrated in Fig. 13. Linear correlation coefficients between these two trials were computed from time series at each grid point. The result shown in Fig. 14b was obtained by taking the median correlation within 126-m range bins. The rms deviations increase dramatically in the low wSNR region (greater range) and the correlations decrease, as one would expect. In (Fig. 14a) the differences are absolute values, so the largest contributions are from the quantities that have the largest magnitudes. The correlations in (Fig. 14b) are normalized, and the curves show that the differences between runs behave similarly in a relative sense for all four variables. In particular the two runs are expected to be highly correlated in the high SNR region, and Fig. 14b shows that this is the case.

Figure 15 shows a comparison of profiles of the resolved-scale variances between trials 38 and 43. The general shapes of these profiles are characteristic of LES results for the CBL (Moeng and Sullivan 1994). Throughout most of the depth of the CBL the profiles are well correlated, but in the low SNR region near the top of the simulated domain, the profiles of all quantities diverge. The equally weighted data produce higher variances in these regions because of a greater contribution from low SNR (noisier) data to the retrieved fields. Figure 16 shows a comparison of the heat-flux profile (resolved plus SGS) between trials 38 and 43. The profile is nearly linear through the depth of the CBL, and as in Fig. 15, the significant discrepancies are mostly near the top in the lower SNR region.

5. Conclusions

In this paper a 4DVAR algorithm for retrieval of wind and thermodynamic fields in the ABL using coherent Doppler lidar data has been described. The retrieval algorithm uses a forward model that simulates dry, shallow incompressible flow with the Boussinesq approximation. The adjoint method is used to find the initialization of the forward model that gives the best fit to radial velocity measurements from a coherent Doppler lidar. Measurements are obtained by repeatedly scanning a 3D volume of the ABL. A system of adjoint equations was developed assuming that the SRS fluxes were given by generic functions of the resolved-scale rates of strain, potential temperature gradient, and potential temperature. These expressions were then specialized using standard gradient transport theory in which the eddy viscosity was assumed to be a prescribed function of height.

Instead of interpolating the radial velocity measurements to the model grid, the approach adopted here involved interpolating the model fields to the observation coordinates. This obviates the need to subject the data to questionable preprocessing since the data are essentially ingested “as is.” It is left for the forward model to optimally interpolate in space and time. Using this approach the error covariance matrix assumes a diagonal structure and the formulation of the cost function is simplified.

One goal of this study was to evaluate the effects of the wSNR-dependent measurement error on the retrieved fields. Radial velocity precision as a function of the wSNR was estimated from time series analysis of real fixed-beam data. This provided a calibration curve that could be used to assign velocity precisions to individual radial velocity measurements.

The retrieval algorithm was applied to real Doppler lidar data collected under afternoon convective conditions during the CASES-99 field program and was shown to produce plausible results. The impact of the wSNR-dependent measurement error was investigated by comparing a retrieval using equally weighted data with a retrieval using the estimated velocity precisions. At near range the fields were well correlated. However, at greater ranges (low wSNR), as the velocity precision exceeds the standard deviation of the measurements, the correlation decreased rapidly. Furthermore, when the measurements were equally weighted, retrievals produced larger variances in all prognostic variables. This is clearly the result of not suppressing the contribution from low wSNR data in the cost function. The calculation of vertical profiles of variances and covariances (fluxes) is a specific example of where equally weighted measurement input produced artificially large values of fluctuating quantities. Because these profiles are expected to be a major product of these retrievals, such discrepancies illustrate the importance of considering measurement quality in the retrieval.

The results produced by the retrieval algorithm are plausible, but obviously it would be highly desirable to verify them against real data. One method is to compare results with data from towers embedded in the retrieval volume. Preliminary comparisons of the retrieval data (from other days than the one presented here) with the 60-m tower data are promising. Preferable to this method would be to compare volume data with an independently sampled volume. A good method for validating the retrievals in this way would involve deploying two Doppler lidars. Both lidars must sample the same volume of space with adequate angular separation between the beams. A straightforward approach would then be to perform a single-Doppler retrieval with one of the systems and compare those results with the radial velocity measurements from the second system. A dataset was recently collected for the purpose of testing this method.

Nevertheless, to have confidence in the results of these retrievals it is important to establish how robust the results are to changes in the various input values that must be prescribed in the model, such as the base-state θ profile. If the model proved very sensitive to this profile, for example, then it would be critical to specify it as accurately as possible, or confidence in the retrieval would be low. On the other hand, if the retrieval proved to be relatively insensitive to the input profile, then one could have confidence in the retrieval even if the input profile were not well known. An investigation of the sensitivity of the retrieval method to many of these factors, including the base-state θ profile, the K(z) formulation, and model resolution, is the subject of Part II.

Acknowledgments

This work was supported by the National Science Foundation (Grant ATM-9908453). Funding for field measurements was also provided in part by the U.S. Army Research Office under Proposals 37522-GS and 43711-EV, and the DOD Center for Geosciences/Atmospheric Research at Colorado State University via Cooperative Agreement DADD19-02-2-0005. We wish to thank Dr. Juanzhen (Jenny) Sun for her consultation on this work and for providing a copy of her code for our reference. We also thank Dr. Yelena Pichugina for assistance with data preparation and display, and members of the Optical Remote Sensing Division at the National Oceanic and Atmospheric Administration, Environmental Technology Laboratory.

REFERENCES

  • Banta, R. M., Newsom R. K. , Lundquist J. K. , Pichugina Y. L. , Coulter R. L. , and Mahrt L. , 2002: Nocturnal low-level jet characteristics over Kansas during CASES-99. Bound.-Layer Meteor, 105 , 221252.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Blumen, W., Banta R. M. , Burns S. P. , Fritts D. C. , Newsom R. , Poulos G. S. , and Sun J. , 2001: Turbulence statistics of a Kelvin– Helmholtz billow event observed in the nighttime boundary layer during the CASES-99 field program. Dyn. Atmos. Oceans, 34 , 189204.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Chai, T., Lin C. L. , and Newsom R. K. , 2004: Retrieval of microscale flow structures from high-resolution Doppler lidar data using an adjoint model. J. Atmos. Sci, 61 , 15001520.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deardorff, J. W., 1972: Numerical investigation of neutral and unstable planetary boundary layers. J. Atmos. Sci, 29 , 91115.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Deardorff, J. W., 1974: Three dimensional numerical study of the height and mean structure of a heated planetary boundary layer. Bound.-Layer Meteor, 7 , 81106.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Frehlich, R. G., and Yadlowsky M. J. , 1994: Performance of mean-frequency estimators for Doppler radar and lidar. J. Atmos. Oceanic Technol, 11 , 12171230.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Frehlich, R. G., Hannon S. M. , and Henderson S. W. , 1994: Performance of a 2-μm coherent Doppler lidar for wind measurement. J. Atmos. Oceanic Technol, 11 , 15171528.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Grund, C. J., Banta R. M. , George J. L. , Howell J. N. , Post M. J. , Richter R. A. , and Weickmann A. M. , 2001: High-resolution Doppler lidar for boundary layer and cloud research. J. Atmos. Oceanic Technol, 18 , 376393.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Harlow, F. H., and Welch J. E. , 1965: Numerical calculation of time-dependent viscous incompressible flow of fluid with free surface. Phys. Fluids, 8 , 21822189.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lenschow, D. H., Mann J. , and Kristensen L. , 1994: How long is long enough when measuring fluxes and other turbulence statistics? J. Atmos. Oceanic Technol, 11 , 661673.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lenschow, D. H., Wulfmeyer V. , and Senff C. , 2000: Measuring second- through fourth-order moments in noisy data. J. Atmos. Oceanic Technol, 17 , 13301347.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Lin, C-L., Chai T. , and Sun J. , 2001: Retrieval of flow structures in a convective boundary layer using an adjoint model: Identical twin experiments. J. Atmos. Sci, 58 , 17671783.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Mayor, S. D., Lenschow D. H. , Schwiesow R. L. , Mann J. , Frush C. L. , and Simon M. K. , 1997: Validation of NCAR 10.6-μm CO2 Doppler lidar radial velocity measurements and comparison with a 915-MHz profiler. J. Atmos. Oceanic Technol, 14 , 11101126.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moeng, C-H., 1984: A large-eddy-simulation model for the study of planetary boundary-layer turbulence. J. Atmos. Sci, 41 , 20522062.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Moeng, C-H., and Sullivan P. P. , 1994: A comparison of shear- and buoyancy-driven planetary boundary layer flows. J. Atmos. Sci, 51 , 9991022.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Newsom, R. K., and Banta R. M. , 2003: Shear-flow instability in the stable nocturnal boundary layer as observed by Doppler lidar during CASES-99. J. Atmos. Sci, 60 , 1633.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Poulos, G. S., and Coauthors, 2002: CASES-99: A comprehensive investigation of the stable nocturnal boundary layer. Bull. Amer. Meteor. Soc, 83 , 555581.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Press, W. H., Flannery B. P. , Teukolsky S. A. , and Vetterling W. T. , 1988: Numerical Recipes in C: The Art of Scientific Computing. Cambridge University Press, 317 pp.

    • Search Google Scholar
    • Export Citation
  • Rye, B. J., and Hardesty R. M. , 1993: Discrete spectral peak estimation in Doppler lidar. I: Incoherent spectral accumulation and the Cramer-Rao bound. IEEE Trans. Geosci. Remote Sens, GE-31 , 1627.

    • Search Google Scholar
    • Export Citation
  • Rye, B. J., and Hardesty R. M. , 1997: Detection techniques for validating Doppler estimates in heterodyne lidar. Appl. Opt, 36 , 19401951.

  • Sun, J., and Crook A. , 1994: Wind and thermodynamic retrieval from single Doppler measurements of a gust front observed during Phoenix II. Mon. Wea. Rev, 122 , 10751091.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, J., and Crook A. , 1997: Dynamical and microphysical retrieval from Doppler radar observations using a cloud model and its adjoint. Part I: Model development and simulated data experiments. J. Atmos. Sci, 54 , 16421661.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, J., and Crook A. , 1998: Dynamical and microphysical retrieval from Doppler radar observations using a cloud model and its adjoint. Part II: Retrieval experiments of an observed Florida convective storm. J. Atmos. Sci, 55 , 835852.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, J., Flicker D. W. , and Lilly D. K. , 1991: Recovery of three-dimensional wind and temperature fields from simulated single-Doppler radar data. J. Atmos. Sci, 48 , 876890.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Sun, J., and Coauthors, 2002: Intermittent turbulence associated with a density current passage in the stable boundary layer. Bound.-Layer Meteor, 105 , 199219.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Troen, I. B., and Mahrt L. , 1986: A simple model of the atmospheric boundary layer: Sensitivity to surface evaporation. Bound.-Layer Meteor, 37 , 129148.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wulfmeyer, V., Randall M. , Brewer W. A. , and Hardesty R. M. , 2000: 2-μm Doppler lidar transmitter with high frequency stability and low chirp. Opt. Lett, 25 , 12281230.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wyngaard, J. C., 1985: Structure of the planetary boundary layer and implications for its modeling. J. Climate Appl. Meteor, 24 , 11311142.

    • Crossref
    • Search Google Scholar
    • Export Citation

APPENDIX A

Adjoint Equations

The adjoint equations are derived by requiring that the first variation of Eq. (12) vanish for t > 0. The discrete model equations are given when the first variation of L with respect to each adjoint variable is set to zero. With those terms eliminated we are left with
i1520-0426-21-9-1328-ea1
As described in appendix B, all variations appearing in (A1) must be rewritten in terms of variations of the model variables, that is, δui, δθ, δp, and δug. After considerable manipulation it is possible to express (A1) in the following form:
i1520-0426-21-9-1328-ea2
Specific expressions for n+1i, n+1, n+1i, and Ũn+1gi are presented in appendix B. Due to the discrete temporal integration scheme, n+1i and n+1 depend on the model variables at t = nΔt and the adjoint variables at t = (n + 1)Δt. The quantities n+1i and Ũn+1gi depend only on the adjoint variables at t = (n + 1)Δt.
Equation (A2) implies that the gradient of L with respect to ug is given by
i1520-0426-21-9-1328-ea3
The gradient of L with respect to the other model variables will vanish for 1 ≤ nN − 1 if ũN = 0, θ̃N = 0, N = 0, and
i1520-0426-21-9-1328-ea4
The last equality implies that (see the appendix B)
2n+1n+1
where n = ∇ · ũn. It can be shown that for n = 0
i1520-0426-21-9-1328-ea8

Equations (A4), (A5), and (A7) define the so-called adjoint model. Equations (A4) and (A5) represent backward time integrations that are solved by starting at n = N − 1, with ũN = 0, θ̃N = 0, and N = 0, and then iterating until n = 0. Equation (A7) is a diagnostic relation for n+1. Since the adjoint equations depend on the prognostic model variables, it is necessary to store the velocity and potential temperature fields from a previous integration of the forward model in order to integrate the adjoint equations. Once the adjoint equations have been solved, the gradient of L with respect to the model's initial state and the geostrophic wind profile can be determined from Eqs. (A3), (A8), and (A9).

The gradient of the cost function with respect to u enters as a forcing term in Eq. (A4). Gradients of Jobs and Jd with respect to u can be computed from Eqs. (8) and (11), respectively. The first variation of Jd is given by
i1520-0426-21-9-1328-ea10
With the proper selection of boundary conditions it can be shown that
i1520-0426-21-9-1328-ea11
Using Eqs. (8) and (9) in (10) the first variation of Jobs is given by
i1520-0426-21-9-1328-ea12
which implies that
i1520-0426-21-9-1328-ea13
Therefore,
i1520-0426-21-9-1328-ea14

APPENDIX B

Adjoint Forcing Terms: General Form

All variations appearing in (A1) must be rewritten in terms of variations of the model variables, that is, δui, δθ, and δp. This involves a fairly straightforward, albeit tedious, application of the chain rule of differential calculus and appropriate selection of boundary conditions for the adjoint variables. For example, consider the term
i1520-0426-21-9-1328-eqb1
that appears in the expression for Σr ũn+1iδFni in Eq. (A1). This term can be written
i1520-0426-21-9-1328-eb1
where n = ∇ · ũn. The first term on the right-hand side of (B1) can be rewritten as
i1520-0426-21-9-1328-eqb2
Since all variables are horizontally periodic, it is clear that the first two terms on the right-hand side of the above expression are zero. Furthermore, the last term is also zero if we require that n = 0 at the top and bottom of the domain. Therefore,
i1520-0426-21-9-1328-eqb3
Thus, terms involving δF, δG, and δP that appear on the right-hand side of Eq. (A1) can be obtained by repeated application of the above procedure. The results are
i1520-0426-21-9-1328-eb2
Collecting terms then gives
i1520-0426-21-9-1328-eb5
where it has been assumed that
i1520-0426-21-9-1328-eqb4

APPENDIX C

Adjoint Forcing Terms: Specific Form

Using Eqs. (20) and (21), expressions for the various partial derivatives of ϕij and ψi that appear in the appendix B become
i1520-0426-21-9-1328-ec1
We further simplify the problem by neglecting the Coriolis force; that is, fc = 0. With these assumptions one can show that Eqs. (B5) and (B6) reduce to
i1520-0426-21-9-1328-ec4
Additionally, Km and Kh are assumed to depend only on height; therefore,
i1520-0426-21-9-1328-eqc1

Fig. 1.
Fig. 1.

Plot of the mean value of the percent deviation of R from unity, 100|R − 1|, as a function of the scaling parameter a. The quantity R is given by Eq. (24) and 100|R − 1| was averaged over 10 realizations of the random vector δ

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 2.
Fig. 2.

Flow diagram of the retrieval algorithm

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 3.
Fig. 3.

Map of the CASES-99 main field site showing the scan coverage (solid line), the model domain boundaries (dotted line), the lidar (HRDL), and the 60-m tower. Within the scan coverage area the terrain elevation varies by about 80 ft (25 m). The lowest point occurs in the gully near the southernmost portion of the scan. The highest point occurs on a ridge just to the north of this gully

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 4.
Fig. 4.

Samples of radial velocity fields used in the assimilation trials. These images show quality controlled radial velocity at an elevation angle of 2° from (a) the first and (b) the second volume scans

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 5.
Fig. 5.

Base-state wind and virtual potential temperature profiles. Base-state virtual potential temperature profile, θb, was obtained from a radiosonde released at about 1900 UTC (1400 CDT) from the main field site. Base-state wind profiles (ub, υb) were computed from the lidar volume scan data using a VAD-type processing technique

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 6.
Fig. 6.

Range–time displays of (a) radial velocity and (b) the logarithm of wSNR. These data were recorded on 19 Oct 1999 as the lidar stared nearly horizontally toward the north. The winds were southerly and relatively strong. The tilted linear features apparent in the radial velocity data were created by gust structures propagating downwind with time

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 7.
Fig. 7.

(a) Individual time series of radial velocity from the data shown in Fig. 6a. Solid line (dotted line) is a time series at a range of 600 m (2400 m) from the lidar. (b) Autocovariance functions (ACFs) of the time series shown in (a)

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 8.
Fig. 8.

Estimated radial velocity measurement precision as a function of wSNR

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 9.
Fig. 9.

Measurement precision fields corresponding to the radial velocity data shown in Fig. 4

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 10.
Fig. 10.

(a) Plot of Jobs normalized by its starting value as a function of iteration for retrieval 49. (b) Distribution of Δm/σm for retrieval 49. The distribution in (b) indicates that 87% of the model generated radial velocities occur within the estimated error of the measured radial velocities

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 11.
Fig. 11.

Horizontal cross sections at z = 100 m for retrieval 49. (a) Perturbation horizontal velocity vector field; regions of horizontal convergence (negative divergence) are shaded. (b)–(d) Perturbation virtual potential temperature, vertical velocity, and vertical vorticity, respectively; shaded areas indicate negative values. Regions labeled by A, B, and C are discussed in the text

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 12.
Fig. 12.

Vertical cross sections at y = 0 m for retrieval 49. (a) Perturbation u and w vector field. (b) Perturbation virtual potential temperature. Shaded areas indicate negative values. Regions labeled by A and C are discussed in the text and correspond to structures observed in the horizontal cross sections shown in Fig. 11

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 13.
Fig. 13.

Comparisons between retrievals 38 and 43. (a), (c) Horizontal cross sections of the perturbation horizontal velocity vector fields for retrieval 38 and 43, respectively; regions of negative vertical velocities are shaded gray. (b), (d) Horizontal cross sections of the perturbation virtual potential temperature for retrieval 38 and 43, respectively; shaded regions indicate negative perturbation virtual potential temperature

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 14.
Fig. 14.

Rms deviations and linear correlation coefficients between retrievals 38 and 43 as functions of range from the lidar. The solid curve in (a) shows the rms deviation between u from retrieval 38 and u from retrieval 43. Rms deviations for υ, w, and θ are shown with the dotted, dashed, and dash–dotted curves, respectively. The solid curve in (b) shows the median linear correlation coefficient between u from retrieval 38 and u from retrieval 43. Median linear correlation coefficients for υ, w, and θ are shown with the dotted, dashed, and dash–dotted curves, respectively

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 15.
Fig. 15.

Vertical profiles of horizontally averaged variances for (a) u, (b) υ, (c) w, and (d) θ. Solid curves are the results for retrieval 38 and the dotted curves are the results for retrieval 43

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Fig. 16.
Fig. 16.

Vertical profiles of horizontally averaged kinematic heat flux for retrieval 38 (solid line) and retrieval 43 (dotted line)

Citation: Journal of Atmospheric and Oceanic Technology 21, 9; 10.1175/1520-0426(2004)021<1328:ACDLMI>2.0.CO;2

Table 1.

HRDL performance characteristics

Table 1.
Table 2.

Tower variances and fluxes, averaged over depth of tower from 2045 to 2115 UTC (1545–1615 CDT) 25 October 1999

Table 2.
Table 3.

Description of trials performed

Table 3.
Save