## 1. Introduction

Advances in remote sensing techniques, such as radar and lidar, allow measurement of high spatial resolution data of the atmospheric boundary layer (ABL) for the study of atmospheric flow structures. For instance, Weckwerth et al. (1997) used the high-resolution Doppler lidar at the Environmental Technology Laboratory of the National Oceanic and Atmospheric Administration, which has about 30-m range resolution and less than 5 cm s^{−1} velocity resolution (Grund et al. 1997), to reveal organized streak structures over flat terrain. Cooper et al. (1997) and Hagelberg et al. (1998) used lidar to identify multiscale Rayleigh–Bernard-like cells in the surface layer of a marine ABL. Nevertheless, three-dimensional (3D) wind vector and temperature data, required for understanding atmospheric flows, are not observed by Doppler radar or lidar.

In recent years, new techniques have been developed to merge limited observations with dynamic models to derive more complete atmospheric data. Among them is four-dimensional variational data assimilation (4DVAR) with the aid of an adjoint model, which has been shown to be promising in meteorological and oceanographic applications (e.g., Talagrand and Courtier 1987; Thacker and Long 1988). A full operational application of the variational approach has not been implemented mainly due to the substantial CPU time required for the optimal solution. However, since the introduction of the method of adjoint equations to perform data assimilation, intensive research has been conducted using models of increasing complexity and dimensions (see, e.g., Talagrand and Courtier 1987; Navon et al. 1992; Rabier and Courtier 1992; Zou et al. 1993; Sun and Crook 1998).

Sun et al. (1991) and Sun and Crook (1994, 1997, 1998) used 4DVAR with a nonhydrostatic numerical model and radar observations to retrieve atmospheric convective-scale flow structures. For instance, they retrieved the wind and thermodynamic fields of a gust front (Sun and Crook 1994), resolving finescale flow structures along the leading edge. Later, Sun and Crook (1997, 1998) added microphysical parameterizations to the technique and successfully retrieved detailed wind, thermodynamics, and microphysics from a simulated convective storm.

More recently, the 4DVAR developed by Sun and Crook (1997, 1998) was implemented in the Washington D.C.–Baltimore Weather Forecast Office to operationally provide low-level wind and temperature information for the National Center for Atmospheric Research (NCAR) Auto-Nowcaster system (Roberts et al. 1998). The applicability of their technique to retrieve microscale turbulent coherent structures in the ABL, however, has to be investigated. If the retrieval of microscale structures is effective and accurate, lidar data can be integrated to provide more detailed atmospheric information for fundamental research and operational application. This could increase the lead time for the forecast of initiation of moist convection, which may evolve into thunderstorms. It could also lead to new algorithms using the derived data for the detection of turbulent coherent structures, microbursts, and wind shear.

The objectives of this paper are twofold. One is to add two new features, a surface (momentum and temperature) flux model and a height-dependent eddy viscosity model, into the 4DVAR developed by Sun et al. (1991) for the retrieval of turbulent structures in the ABL. In large-eddy simulation (LES) of high Reynolds number ABL flows, flow near rough terrain is not well resolved and a surface flux model is needed to ensure proper realization of flow structures in these regions (Sullivan et al. 1994). Subgrid-scale (SGS) eddy viscosity is often parameterized to correlate with SGS turbulence kinetic energy (TKE), which varies in time and space. Use of a height-dependent eddy viscosity allows assessment of eddy viscosity effect on retrieval quality.

The other objective is to test the sensitivity of retrieval quality to the initial guess, the observational frequency, the observational error, the surface flux model, the eddy viscosity and diffusivity, and the temporal and spatial smoothness penalty functions. This information is crucial to the application of the 4DVAR to real data in the future. To achieve this second objective, a sequence of numerical “identical twin” experiments (Long and Thacker 1989) are carried out. The 4DVAR technique involves forward integration of numerical prediction model equations in time and backward integration of adjoint equations. In identical twin experiments, observational data generated by the 4DVAR prediction model are assimilated back into the 4DVAR. Without degrading the synthetic observational data, these experiments tend to generate overoptimistic results. Thus we shall degrade the observational data by reducing data availability, adding observational errors, and altering boundary conditions.

The paper is organized as follows. The numerical method is discussed in section 2. In section 3, the procedure for generating observational data is described. Results are presented in section 4, and conclusions in section 5.

## 2. Numerical method

### a. Prediction model

*U*

_{1},

*U*

_{2}, and

*U*

_{3}(

*U,*

*V,*and

*W*) are velocity components in the respective

*x*

_{1},

*x*

_{2}, and

*x*

_{3}(

*x,*

*y,*and

*z*) directions. Spatial variables

*x*

_{1},

*x*

_{2}, and

*x*

_{3}correspond to the east, north, and vertical directions, respectively. Here

*θ,*Θ, and Θ

_{0}are the fluctuating, background (function of

*z*only), and reference virtual potential temperature, respectively. Lowercase variables denote fluctuating parts of the variables. Eddy viscosity

*ν*and thermal diffusivity

*κ*are assumed constant in the SDPR. A second-order finite volume method is applied for spatial differencing and a second-order Adam–Bashforth method is used for time-advancement of dependent variables, which are arranged on a staggered, orthogonal grid. Mass conservation is enforced by solving a pressure-Poisson equation derived from the continuity equation (1) and momentum equations (2).

*ν*and thermal diffusivity

*κ*are made functions of height to consider the increasing turbulence intensity as the surface is approached. The vertical profiles of

*ν*and

*κ*are generated by the NCAR-LES code (Moeng 1984; Sullivan et al. 1994), which solves a prognostic SGS TKE equation for the SGS turbulence energy

*e*and computeswhere

*l*is a mixing length and Δ an average grid spacing.

*i*is either 1 or 2 and the angle bracket denotes spatial averaging over an

*x*−

*y*plane. Here Θ

_{1}is the background virtual potential temperature at the first vertical grid point and Θ

_{s}denotes the surface temperature.

*U*

_{i}and

*θ*are data at the first vertical grid level. To evaluate the mean surface momentum 〈

*τ*

_{i3}〉 and temperature 〈

*τ*

_{θ3}〉 fluxes, the surface-layer friction velocity

*u*∗ and temperature scale

*θ*∗ (defined as −

*u*

_{3}

*θ*

*u*∗) are estimated using the similarity equations,where

*k*is the von Kármán constant,

*L*the Monin–Obukhov length scale,

*z*

_{o}the surface roughness height, and

*z*half the vertical grid spacing. Here Ψ

_{M}and Ψ

_{H}are given bywhere

*ξ*= [1 − (15

*z*/

*L*)]

^{1/4},

*η*= [1 − (9

*z*/

*L*)]

^{1/2}, and

*ζ*=

*π*/2 = 1.57079633. Mean surface momentum and temperature fluxes are then calculated using

Lateral boundary conditions for *U,* *V,* *W,* and *θ* fields are obtained by linearly interpolating the observational data. The gradient-free boundary condition is imposed at the top of the domain for *U,* *V,* and *θ* fields. For field *W* the Dirichlet boundary condition *W* = 0 is used. Additional terms and features, such as Coriolis force and a Galilean transformation of the coordinates with a constant speed, are added to the model equations. The Galilean transformation mimics a moving grid and is applied to reduce the maximum velocity over the domain, thereby increasing the time step allowed and reducing computer memory usage. The 4DVAR method requires huge amounts of computer memory, depending on the grid size, the assimilation period, and the time step. Variable management in code is also optimized, reducing memory usage about twofold as compared with the SDPR.

### b. Optimization method

*J*is defined aswhere

*U*

_{rad}and

*U*

^{obs}

_{rad}

*α*is the validity coefficient indicating data quality and is taken as unity in the study. The second term, a penalty function, represents the nondivergent constraint for the initial velocity field. With the stipulation that this penalty term cannot dominate the cost function, the value for coefficient

*β*should be selected as large as possible to obtain a divergence-free initial velocity vector field, and is set to 100. As a consequence, the second term in Eq. (13) is about 1% of the first term after 100 iterations and about 15% after 400 iterations. This cost function does not include a background term measuring the difference from the previous forecast because the aim of the study is to assess the application of the 4DVAR to retrieval of microscale turbulent eddies, not forecast. The relationship between the radial velocity and the three velocity components is expressed aswhere (

*x*

_{1o},

*x*

_{2o},

*x*

_{3o}) are the radar (or lidar) coordinates.

*λ*

_{i}or adjoint variables) are appended to the cost function [equation (13)]. This leads to the Lagrange function,

*L*with respect to

*U,*

*V,*

*W,*

*θ,*

*P,*

*λ*

_{u},

*λ*

_{υ},

*λ*

_{w},

*λ*

_{θ}, and

*λ*is equivalent to the constrained minimization of

_{p}*J*with respect to

*U,*

*V,*

*W,*

*θ,*and

*P.*The first variation of

*L*with respect to

*λ*

_{u},

*λ*

_{υ},

*λ*

_{w},

*λ*

_{θ}, and

*λ*

_{p}restores the governing equations. The first variation of

*L*with respect to

*U,*

*V,*

*W,*

*θ,*and

*P*gives adjoint equations for

*λ.*Backward integration of the adjoint equations yields

*λ*

_{u},

*λ*

_{υ},

*λ*

_{w}, and

*λ*

_{θ}at the initial state, corresponding to the initial

*U,*

*V,*

*W,*and

*θ*gradients,With these gradients, the limited-memory quasi-Newton algorithm BFGS (Liu and Nocedal 1989) is applied to find the optimal initial guess for the prediction model, whose solution best fits the observations in a least squares sense.

Recently other minimization methods have been proposed to accelerate convergence, such as the adjoint Newton algorithm (Wang et al. 1997) and the quasi-inverse method (Kalnay et al. 2000). The quasi-inverse method is a generalization and simplification of the adjoint Newton algorithm. Both methods can be considered as variants of the Newton algorithm, which converges in a single iteration for a quadratic function. Unlike the Newton algorithm, both methods integrate the tangent linear model backward in time instead of computing the Hessian or the gradient of the cost function. It was reported that these methods are many times faster than the quasi-Newton BFGS method for certain applications. One of the major restrictions of these methods is the requirement that the observation operator (the operator that transforms the model variables to the observational variables) is invertible, which is impossible in the case of radar and lidar data assimilation due to the fact that the model variables are much more than the observed variables. In addition, the backward integration of the tangent linear model has to reverse the sign of the diffusion terms to avoid numerical instability. Its effect on the retrieval of microscale turbulent structures requires further study. Another concern is about finding a “complete suitable final condition” for the backward integration of the tangent linear model. One has to recognize that radar and lidar data are measured in a continuous time sequence. It is not possible to have a complete volume data at a single time. Despite of the limitations of the methods, it is possible to use them as a preconditioner for the current model to provide a good first guess after some approximations as discussed in Kalnay et al. (2000).

Two averaging approaches are used in the calculation of mean momentum and temperature fluxes [the angle brackets in Eqs. (7)–(12)]. One approach applies spatial *x* − *y* plane averaging at every time step of forward integration using data from the previous step. The other approach applies spatial and temporal averaging at the beginning of each new iteration using data from the previous iteration. Recall that every iteration involves forward integration of the prediction equations and backward integration of the adjoint equations. Variations of the mean fluxes are assumed negligible as compared with those of instantaneous variables. Thus the adjoints of instantaneous surface momentum and temperature fluxes [Eqs. (5) and (6)] can be derived in a straightforward manner.

## 3. Observational data

Observational data are generated using the prediction model previously described. Periodic boundary conditions are imposed in the *x* and *y* directions as in most LES of the ABL, while in retrieval experiments the more realistic Dirichlet lateral boundary conditions are imposed. The initial conditions for dependent variables are obtained using the NCAR-LES code (Moeng 1984; Sullivan et al. 1994). A grid size of 48 × 48 × 48 is used on a computational domain of 5 km × 5 km × 2 km with 104-m horizontal and 42-m vertical spatial resolution. The spatial resolution is chosen as typical of lidar range. For instance, the range resolution for the NOAA mini-MOPA CO_{2} Doppler lidar is 100 m and its maximum range is 5 − 15 km. The NOAA–ETL high-resolution Doppler lidar can measure radial air motion with 30-m resolution. The simulated CBL flow is driven by a geostrophic wind of 10 m s^{−1} and a temperature flux of 0.24 K m s^{−1}. A capping inversion layer is imposed at about *z* = 980 m. The Coriolis parameter *f* is 10^{−4} 1 s^{−1} and the roughness height *z*_{0} is 0.16 m. The stability parameter −*z _{i}*/

*L,*where

*z*

_{i}is the averaged CBL height and

*L*is the Monin–Obukhov length, is about 15.

The simulation was first carried out for about 1.56 h (real time elapsed for turbulence to evolve) using the NCAR-LES code, optimized for simulation of ABL flows. Using the large-eddy turnover time *z*_{i}/*w*∗ ≈ 490 s as a reference, where *w*∗ is the free-convection scaling velocity, there are about 11 convective eddies sequentially reaching the top of the CBL from the surface during the 1.56-h period. The generated velocity and temperature fields together with the *x* − *y* plane-averaged eddy viscosity and diffusivity were input to the prediction model described in section 2. To reduce computer memory usage, the vertical grid number was reduced from 48 to 45, corresponding to a physical vertical extent of 1 875 m. The simulation was continued with a time step of 5 s for 50 s and 3D data were subsequently recorded every 25 s for 300 s, generating 13 3D instantaneous datasets. The Coriolis force had negligible effect on data generated over 5 min. The calculated velocity fields were then converted to “observed” radial velocity fields [*U*^{obs}_{rad}*x,* *y,* *z*) = (0,0,20.8) m.

The time required for a lidar volume (3D) scan depends on several factors, such as the scanning rate, the number of pulses averaged, and the pulse repetition frequency (PRF). At PRF = 200 Hz with a 100 pulse average, a data beam is generated every 0.5 s. At a scanning rate of 10° s^{−1}, an angular resolution of 5° can be obtained. A volume scan is composed of a series of plane-position indicator (PPI) scans at several elevation angles. Since each PPI scan sweeping a 90° angle takes about 18 s at this rate, it is impossible to measure a volume dataset every 25 s (the time interval of the recorded datasets). For the following experiments, we first used the 13 radial velocity datasets as input, representing the best retrieval this 4DVAR model can achieve. These datasets were then used to construct two volume scan datasets measured in 5 min, emulating the lidar scanning feature.

Since the 3D velocity and temperature fields calculated at each time step during forward integration of the prediction model must be stored in computer memory for later use in backward integration of the adjoint equations, a large amount of memory is required. For instance, with a 48 × 48 × 45 grid, 5-s time step, and 300-s period, each dependent variable requires about 6.3 M Words of memory and each run about 90 M Words. The original SDPR requires more than 160 M Words for management of validity coefficients *α* and *β* in Eq. (13) and other penalty functions. Memory usage can be further reduced by temporarily storing numerical data on a hard disk, but that reduces model efficiency with excessive read–write operations during the search for the optimal solution.

The mean velocity and temperature profiles of the “simulated” observational data and their variances are displayed in Figs. 1a and 1b, respectively. The top of the CBL is capped by an inversion layer, characterized by the large gradient of mean virtual potential temperature shown in Fig. 1a. Typical profiles of vertical velocity variance in the CBL as measured by Lenschow et al. (1980) for various stability regions are shown for comparison. Although the dependence of these profiles on stability parameter −*z*_{i}/*L* is not clear, our data agree well with field data.

## 4. Results

*N*experimental data points (

*x*

_{i},

*y*

_{i}), where the subscript

*i*refers to the

*i*th dataset. The coefficients

*A*and

*B*in the linear relationship,

*y*

*Ax*

*B,*

*x*

*y*

*R*

_{F}

*exact*denotes the exact data, and “rms” means root-mean-square. Correlation coefficients and rms errors of retrieved data averaged throughout the boundary layer at the middle of the assimilation time window after 50 iterations of each experiment are tabulated in Table 2.

### a. Initial guess

Consider experiments using 13 3D datasets as observations and updating mean surface fluxes at every time step (cases 1–5 in Table 1). Cases 1, 2, and 3 use velocity profiles displayed in Fig. 2 as the initial guess. Case 1 adopts the mean velocity profiles 〈*U*_{i}〉 averaged over 13 observations; case 2, the same profiles with magnitudes reduced by 20%; case 3, the same profiles containing 20% uniformly distributed random errors expressed by (1 + 0.2*ϵ*) 〈*U*_{i}〉, where |*ϵ*| ≤ 1. Figure 3 displays the evolution of *R _{Ui}*

*R*

_{θ}averaged throughout the boundary layer for these cases as a function of iteration number (NTER). The distribution of

*R*

_{V}is similar to

*R*

_{U}and not shown. The correlation coefficients at NTER = 0 for all variables for all three cases fall between 0.30 and 0.40. After 50 iterations,

*R*

_{Ui}*R*

_{θ}exceed 0.90 and 0.70, respectively, and continue to increase slowly with iterations, suggesting that a perturbation to initial guess tends to decay. Here

*R*

_{θ}is lower than

*R*

_{Ui}The first guess can be obtained alternatively using the velocity tracking technique (Tuttle and Foote 1990). Cases 4 and 5 examine this type of initial guess. Assume that horizontal velocity components with a spatial resolution of 417 m are derived from tracking techniques (i.e., data at every fourth grid point in the *x,* *y* directions). Horizontal velocity components at other grid points can be estimated by applying bilinear interpolation. A 20% random error denoted by *u*_{error} is added to these components. Adopting a no-slip bottom boundary condition, the vertical velocity component is calculated, level by level, using the continuity equation from the surface to the top of the domain. It was expected that this initial guess would lead quickly to the optimal solution; instead, the 4DVAR model became unstable at the very first iteration. The derived vertical velocity component at the top of the CBL became so large that the time step had to be reduced to ensure numerical stability. The excessively strong vertical velocity is attributed to error accumulation from the surface to the top of the CBL through the use of the continuity equation. Reduction in the time step for the 4DVAR model is not desirable due to computer memory requirements. A proposed remedy is to scale down the magnitude of initial velocity components by a factor of *u*_{scale}. To guarantee convergence, *u*_{scale} is found to be less than 0.8.

Table 2 shows that case 4 with *u*_{scale} = 0.5 and case 5 with *u*_{scale} = 0.3 generate better results than case 1, which uses mean velocity profiles as the first guess. The larger the scaling factor *u*_{scale} is, the better the results are. But a too large *u*_{scale} causes numerical instability. Thus, the initial guess for case 5 is adopted for subsequent experiments presented.

### b. Observational frequency

Our attention now turns to the effect of observational frequency on retrieval. As discussed in section 3, it is impossible to measure an “instantaneous” 3D dataset in 25 s. Hence we first reduce observed datasets to three with an interval of 2.5 min and repeat the experiment (case 6). In order to emulate the lidar scanning feature, various strategies utilizing the 13 3D datasets are exercised. As illustrated in Fig. 4, each of the 13 datasets provides only a few *x* − *y* planes of data represented by *s _{i}* at any instant, in contrast to case 1 using all available datasets as observations (only three datasets are shown in Fig. 4 due to space constraint). Each slice of data

*s*

_{i}is composed of several

*x*−

*y*plane data at vertical grid levels

*iz*ranging from

*m*to

*n*denoted by “

*iz*=

*m*−

*n.*” Case 7 uses three volume scan data with a scanning sequence of [

*s*

_{1}(

*iz*= 1 − 6),

*s*

_{2}(

*iz*= 7 − 12),

*s*

_{3}(

*iz*= 13 − 18),

*s*

_{4}(

*iz*= 19 − 24),

*s*

_{1}(

*iz*= 1 − 6),

*s*

_{2}(

*iz*= 7 − 12),

*s*

_{3}(

*iz*= 13 − 18),

*s*

_{4}(

*iz*= 19 − 24),

*s*

_{1}(

*iz*= 1 − 6),

*s*

_{2}(

*iz*= 7 − 12),

*s*

_{3}(

*iz*= 13 − 18),

*s*

_{4}(

*iz*= 19 − 24),

*s*

_{5}(

*iz*= 25 − 45)] provided by observation (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13), respectively. Observation

*I*is taken at [(

*I*− 1) × 25 +

*T*] s, where

_{o}*T*is the time of observation 1. The time interval of these observations is 25 s. Since turbulence intensity above the inversion layer (

_{o}*iz*= 24) is weak, little information is needed there. Except for slice

*s*

_{5}above the inversion layer, data for each of the slices

*s*

_{i}are available at three different times, thereby referred to as “three volume scan data.”

In like manner, for case 8 two volume scan data are constructed with the sequence of [*s*_{1}(*iz* = 1 − 4), *s*_{2}(*iz* = 5 − 8),*s*_{3}(*iz* = 9 − 12), *s*_{4}(*iz* = 13 − 16), *s*_{5}(*iz* = 17 − 20), *s*_{6}(*iz* = 21 − 24), *s*_{1}(*iz* = 1 − 4), *s*_{2}(*iz* = 5 − 8), *s*_{3}(*iz* = 9 − 12), *s*_{4}(*iz* = 13 − 16), *s*_{5}(*iz* = 17 − 20), *s*_{6}(*iz* = 21 − 24), *s*_{7}(*iz* = 25 − 45)] provided by observation (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13), respectively. Radial velocity data at *s*_{i} with *i* = 1 to 6 are available at two different times, and referred to as “two volume scan data.”

Cases 6, 7, and 8 have the same starting point because of the same first guess. After 50 iterations *R*_{U} and *R*_{V} for the three cases are higher than 0.93; *R*_{W} is higher than 0.92 for cases 6 and 7, and is 0.89 for case 8 (Table 2). Figure 5 reveals that the vertical profiles of *R _{Ui}*

*R*

_{F}

*ϵ*

_{θ}for cases 6, 7, and 8 are 0.20, 0.21, and 0.23 K, respectively (Table 2). Recall that case 6 uses three instantaneous 3D datasets, case 7 three volume scan data, and case 8 two volume scan data. A comparison with case 5, which uses the full 13 sets of data and whose

*ϵ*

_{θ}= 0.20 K, indicates that temperature retrieval is less sensitive to observational frequency because observations contain no temperature information. The minimum

*R*

_{θ}is found around

*z*/

*z*

_{i}≈ 0.84 (Fig. 5) where the minimum temperature variance is located (Fig. 1b). Figure 6 shows the vertical profiles of the rms errors of data retrieved from case 8. The distributions roughly parallel those of variances shown in Fig. 1b, indicating that the rms error of retrieved data is approximately proportional to turbulence intensity.

To explore the optimal solution obtainable with two volume scan data, the number of iterations for case 8 is increased to 400. The resulting *R*_{U}, *R*_{V}, *R*_{W}, and *R*_{θ} averaged throughout the boundary layer are 0.97, 0.97, 0.94, and 0.72, respectively; *ϵ*_{U}, *ϵ*_{V}, *ϵ*_{W}, and *ϵ*_{θ} are 0.22, 0.24, 0.29 m s^{−1}, and 0.21 K. These numbers are quite satisfactory considering the lack of temperature information. Figures 7 and 8 show the retrieved instantaneous velocity vector and temperature at three selected heights after 400 iterations. The exact data used to compute radial velocity are also displayed for comparison. Several convergence lines (updraft motions) and divergence regions (downdraft motions) are discernible near the surface. Strong convergence lines are spatially correlated with strong temperature fluctuation in Fig. 8. The temperature field is well retrieved only in the lower part of the CBL (Figs. 8a1 and 8b1). Some updrafts extend higher up into the mixed layer and then are capped by the inversion layer, creating a diverging velocity vector field near the top of the CBL (Fig. 7c1).

We next examine the sensitivity of retrieval quality to vertical range resolution because PPI scans could have various elevation angles. Case 9 uses *x* − *y* plane date at every other vertical grid level and adopts the scanning sequence of [*s*_{1}(*iz* = 1, 3), *s*_{2}(*iz* = 5, 7), *s*_{3}(*iz* = 9, 11), *s*_{4}(*iz* = 13, 15), *s*_{5}(*iz* = 17, 19), *s*_{6}(*iz* = 21, 23), *s*_{1}(*iz* = 1, 3), *s*_{2}(*iz* = 5, 7), *s*_{3}(*iz* = 9, 11), *s*_{4}(*iz* = 13, 15), *s*_{5}(*iz* = 17, 19), *s*_{6}(*iz* = 21, 23), *s*_{7}(*iz* = 25, 27, 29, 31, 33, 35, 37, 39, 41, 43, 45)] provided by observation (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13), respectively. Here *s*_{i}(*iz* = *m* − *n*) means that *iz* varies from *m* to *n*; *s*_{i}(*iz* = *m,* *n*) indicates *iz* = *m* and *n.* The values of *R*_{U}, *R*_{V}, *R*_{W}, and *R*_{θ} at 50 iterations for case 9 are 0.91, 0.92, 0.85, and 0.65, respectively. They are 0.93, 0.94, 0.89, and 0.66 for case 8, which uses twice the data density in the vertical direction. Data with reduced vertical spatial resolution appear to give satisfactory retrieval. It is probably because the vertical length scale is characterized by large-scale thermals capped by the inversion layer. Reduced vertical data points still adequately capture large-scale thermal motions.

### c. Observational error

In this section we study the effect of observational error on retrieval accuracy. The absolute error associated with lidar depends on several factors, such as the number of pulses averaged and the range. As a rule of thumb, radial velocity estimation is considered good so long as the wideband signal-to-noise ratio (SNR) is greater than −10 dB (R. Newsom, NOAA, 1999, personal communication).

For case 10, a random error in the range of ±0.5 m s^{−1} with an rms value of 0.29 m s^{−1} is added to the radial velocity data throughout the computational domain. The selected value roughly corresponds to typical measurement error averaging 100 lidar pulses in a 3-km range on a day with very clear air. The scanning strategy of case 8 (two volume scan data in 5 min) is adopted for subsequent cases. Figure 9 shows correlation coefficients averaged throughout the boundary layer versus number of iterations for cases 8 and 10. In contrast to cases without random errors where correlation coefficients increase monotonically with iteration to asymptotes, the correlation coefficients for case 10 first increase rapidly reaching maxima of *R*_{U} = 0.89, *R*_{V} = 0.90, *R*_{W} = 0.81, and *R*_{θ} = 0.60 at NTER = 30, then drop to 0.80, 0.82, 0.72, and 0.51 at NTER = 200. The degraded data indicates that the error added alters the shape of the cost function and its minimum as well. If the first guess is far from the minimum, the improved solution will first approach both true and altered minima, then deviate from the true minimum as it approaches the altered minimum. Hence as iterations increase, correlation coefficients decrease. The vertical distributions of these coefficients (Fig. 10) indicate that *R*_{U} and *R*_{V} still retain a high value of 0.90 throughout the boundary layer, *R*_{W} drops about 0.10 everywhere retaining a high value of 0.85 in the region of 0.1 ≤ *z*/*z*_{i} ≤ 0.8, and *R*_{θ} is reduced by about 0.08 near the surface.

The effect of error amplitude on retrieval quality is next investigated. Figure 11 displays the amplitude of random error against correlation coefficients and rms errors of retrieved data at 50 iterations. The horizontal velocity components are always more accurate than the vertical velocity component, perhaps because the error added is small relative to the (total) horizontal velocity components but large relative to the vertical velocity component, whose mean value is essentially zero (Fig. 1a). The rms error of retrieved temperature varies from 0.24 to 0.36 K and is less sensitive to error amplitude.

The above sensitivity test offers an opportunity to address the issue regarding the condition of the minimization problem, which characterizes the sensitivity of its solution (retrieved data) with respect to perturbations in the observational data. Mathematically the condition number that depends on the problem as well as the observational data is used to determine whether the problem is well-posed or ill-posed. For large-scale minimization problems, it is impossible to compute the condition number. However, random errors added to the observational data can be regarded as perturbations to the data. Since an ill-posed problem with a very large condition number tends to produce unbounded error in its solution upon small perturbations added to the data, we can assess the condition of our retrieval experiments from the retrieved data. Figure 11 shows that with the largest error amplitude added, 1.5 m s^{−1} (slightly higher than the maximum velocity fluctuation 1.2 m s^{−1}), the maximum rms error of the retrieved data *ϵ*_{W} is about 1.10 m s^{−1} and *R*_{W} is around 0.60. We argue that even with large perturbations to the observational data, the errors in the retrieved data are bounded, so our minimization problems are well-posed.

Since observational error can be spatially correlated, the errors added to cases 11, 12, and 13 are, respectively, vertically, horizontally, and both vertically and horizontally correlated. For case 11, the random error is applied to the *x*–*y* plane data at every fourth vertical grid level and then linearly interpolated to the other points. The linear interpolation serves to correlate errors in space. For case 12, the random error is applied to the *y*–*z* plane data at every fourth grid point in the *x* direction and then interpolated to the other points. In the same fashion for case 13, the random error is added to every fourth grid point in the respective *x,* *y,* *z* directions. The error amplitudes for these cases are adjusted to produce the same rms error of 0.29 m s^{−1}; they are 0.6, 0.6, and 0.86 m s^{−1} for cases 11, 12, and 13. The retrieval sensitivity of a mesoscale collapsing cold pool to the error in the velocity data was studied by Sun and Crook (1996). They found that the adjoint model performs equally well for random and spatially correlated errors.

The evolution of correlation coefficients with iteration for these cases resembles that in case 10 and is not shown. Their average correlation coefficients and rms errors are listed in Table 2. Roughly, case 10 yields about the same retrieval quality as case 12, case 11 has about the same results as case 13. Data retrieved from cases 11 and 13 are less accurate than those from cases 10 and 12. In view of cases 11 and 13 containing vertically correlated errors, the effect of correlated errors is probably more pronounced in the vertical direction than in the horizontal. This phenomenon may be related to the characteristic length scales of turbulent eddies in the horizontal and vertical directions. The vertical length scale of energy-containing updrafts and downdrafts is about the CBL height (980 m), which is larger than the length scale of the vertically correlated error (168 m, 4 grid points). In contrast, the spectrum of the horizontal length scale is much wider due to the periodic homogeneous horizontal physical domain and combined shear-buoyancy effect. For instance, the width of elongated updrafts near the surface (Fig. 7a1) may occupy less than four grid points, the grid size of the correlated error. A consequence is that eddy structures of high spatial frequency do not effectively sense the low-frequency noise, namely, the horizontally correlated error. The accuracy of retrieved data can be further improved by imposing temporal and spatial penalty constraints, so the effect of observational error will be revisited in section 4f.

### d. Surface flux model

Two approaches are applied to calculate mean surface fluxes using Eqs. (7)–(12) (section 2). One approach is to update mean surface momentum and temperature fluxes at each time step using data from the previous time step of the current iteration (denoted by ISUFT = 1). The other approach is to average data retrieved from the immediately previous iteration at the beginning of each new iteration (denoted by ISUFT = 2). Accordingly mean flux calculation is lagging behind. Cases 1–13 adopt ISUFT = 1, case 14 ISUFT = 2. The correlation coefficients and rms errors for case 8 with ISUFT = 1 and case 14 with ISUFT = 2 (Table 2) show that the two methods lead to almost identical results. Case 15, which uses the gradient-free bottom boundary condition of the original SDPR system, shows degraded retrieved temperature data (Table 2). In Fig. 12, the quality of the retrieved temperature deteriorates as the surface is approached.

Case 16 uses Eqs. (5) and (6) with constant momentum and temperature fluxes specified prior to assimilation, denoted by ISUFT = 0. From the simulated CBL we have mean flux values: 〈*τ*_{13}〉 = −0.3618 m^{2} s^{−2}, 〈*τ*_{23}〉 = −0.0378 m^{2} s^{−2}, and 〈*τ*_{θ3}〉 = 0.24 K m s^{−1}. This case produces results almost the same as case 14. Case 17 uses the Dirichlet bottom boundary condition for horizontal velocity components, specifying *U* = *V* = 0 m s^{−1} at the surface. This type of boundary condition allows production of surface fluxes proportional to the vertical velocity gradient. Figure 12 shows that the no-slip (Dirichlet) bottom boundary condition produces better results than the gradient-free (Neumann) condition. Data retrieved using both bottom boundary conditions, however, are less accurate than with the surface flux model.

### e. Eddy viscosity and diffusivity

The LES technique requires a sophisticated SGS model to produce good turbulence statistics. The physics of SGS motions and their interactions with the resolved-scale motions, however, are far from being understood (Lin 1999) and its parameterizations have room for improvement. The primary difference between the 4DVAR and the LES is that the former is driven by observational data of short duration, while the latter is driven by the model until reasonable turbulence statistics are produced. Implementation of a sophisticated SGS model into the 4DVAR requires more memory and computing time. Designing a physics-based SGS model is challenging and whether the 4DVAR requires a sophisticated SGS model is an open question. In this section, results from a series of retrieval experiments based on the height-dependent eddy viscosity model are presented below to shed light on this issue.

Figure 13 shows the vertical profiles of eddy viscosity *ν* and diffusivity *κ* used to generate observational data. The increasing intensity of shear-generated turbulence near the surface reflects on increasing eddy viscosity. The turbulent Prandtl number in the mixed layer is about 1/3. All retrieval experiments presented so far use these profiles in calculation. For case 18, *ν* and *κ* are increased by a factor of two from the original profiles. For case 19, they are reduced by a factor of two. A decrease in eddy viscosity and diffusivity (case 19) leads to lower correlation coefficients and larger rms errors than in case 8. An increase (case 18) surprisingly leads in the opposite direction (Table 2). For case 20, *ν* and *κ* are increased by a factor of 4 and temperature retrieval is poor. It is perhaps because the too strong *ν* and *κ* smooth out finescale turbulent structures. Thus retrieval quality can be improved only if *ν* and *κ* are increased within a certain range.

The retrieval sensitivity to the turbulent Prandtl number *Pr*_{t} is investigated by increasing or decreasing *ν* or *κ* by a factor of two (cases 21, 22, 23, and 24 in Table 1). A comparison of correlation coefficients and rms errors for these cases (Table 2) shows in case 23 that increasing *κ* alone generates the best retrieved data, nearly as good as those using 2*ν* and 2*κ* (case 18). It implies that retrieval quality is less sensitive to error in *ν* approximation and more sensitive to *κ.* The turbulent Prandtl number does not seem critical. Why an increase in eddy diffusivity yields more accurate data will be clarified in the next section.

Cases with height-dependent eddy viscosity and diffusivity are also compared with those with constant ones. Since turbulence intensity above the boundary layer is weak, the values of *ν* and *κ* for case 25 are obtained by averaging those above and below *z*/*z*_{i} = 1. The profiles of mean *ν* and *κ* denoted by *ν*_{m} and *κ*_{m} are displayed in Fig. 13. Data recovered using *ν*_{m} and *κ*_{m} (case 25) are almost as accurate as their height-dependent counterpart (case 8). Doubling the values of *ν*_{m} and *κ*_{m} gives case 26 better retrieved data. Also tested are cases with 2*ν*_{m} and *κ*_{m} and with *ν*_{m} and 2*κ*_{m}. The former yields results nearly identical to case 25 and the latter to case 26. These results indicate that improved retrieval accuracy is essentially attributed to an increase in eddy diffusivity *κ,* consistent with findings for height-dependent *ν* and *κ.* The implication is that a sophisticated SGS model may have limited effect on retrieval quality.

### f. Penalty function

*i,*

*j,*

*k*are the running indices in the respective

*x,*

*y,*

*z*directions. Determination of the penalty constants

*γ*and

*ζ*is empirical and requires experience. However, one basic requirement is that the penalty terms cannot dominate the cost function. Here

*γ*

_{u}=

*γ*

_{υ}= 0.1,

*γ*

_{w}= 0.5, and

*γ*

_{θ}= 1.0 were chosen as temporal smoothness constants, and

*ζ*=

_{u}*ζ*=

_{υ}*ζ*= 0.00005 and

_{w}*ζ*= 0.001 as spatial smoothness constants. The cases presented below include observational error as described in section 4c.

_{θ}Case 27 applies spatial smoothness only and includes random observational error. Parameters for case 28 are those for case 27 except that eddy diffusivity is increased by a factor of 2. For case 29, both temporal and spatial smoothness penalty terms are employed. The following conclusions are based on correlation coefficients and rms errors listed in Table 2. First, smoothness penalty terms do improve retrieved data quality. Second, doubling eddy diffusivity further improves retrieval quality. Due to a lack of temperature information, the doubled eddy diffusivity may act as a spatial smoother, exchanging information with nearby nodes. Third, the spatial smoothness constraint is more effective than the temporal one in improving retrieval accuracy. Recall that all the retrieval experiments were carried out in a moving reference frame to increase the numerical stability limit. As a result, turbulent structures are advected slowly in the reference frame, reducing the time rate change of dependent variables at fixed spatial points. This may explain why temporal smoothness is less effective in experiments.

To examine the effect of error amplitude, experiments shown in Fig. 11 are repeated by imposing temporal and spatial smoothness penalty constraints. Figure 14 shows that smoothness penalty constraints considerably improve retrieval quality over a wide range of error amplitude. The relationship between the rms errors of retrieved velocity and the error amplitude appears to be linear. The rms error of retrieved temperature remains approximately constant at about 0.2 K regardless of error amplitude.

Experiments containing spatially correlated errors (cases 11, 12, and 13) are next revisited by applying both temporal and spatial smoothness penalty constraints. These are designated as cases 30, 31, and 32. They have the same rms observational error as case 29, requiring error amplitudes to be 0.50, 0.60, 0.60, and 0.86 m s^{−1} for cases 29, 30, 31, and 32. The correlation coefficients and rms errors for these cases differ significantly from those for cases 10, 11, 12, and 13 without smoothness penalty terms, showing that smoothness penalty constraints improve accuracy of retrieved data in the presence of various sources of error. With random error, case 29 yields the best results. Cases with vertically (case 30) or horizontally (case 31) correlated errors recover velocity and temperature fields of comparable quality. With observational errors correlated in vertical and horizontal directions, case 32 yields the least accurate results. This may be due to the large error amplitude employed to attain the rms observational error of other cases.

## 5. Conclusions

The 4DVAR model developed by Sun et al. (1991) had been demonstrated as capable of retrieving mesoscale atmospheric flow structures. We have further implemented two new features into this model to retrieve microscale turbulent structures in the atmospheric boundary layer for fundamental research and operational application. One feature, the surface flux model, produces appropriate momentum and temperature fluxes at the first vertical grid level based on Monin–Obukhov similarity theory. The other feature allows eddy viscosity and diffusivity to vary with height, taking into account the effect of varying turbulence intensity with height. The study attempts to assess the applicability of the model to retrieve microscale turbulent structures in the CBL; determine the value of model control parameters for optimal retrieval, including observational frequency, number of iterations, and temporal and spatial smoothness coefficients; and evaluate the relationship between retrieval quality and data error. We have carried out a number of identical twin experiments whose observational data are generated by the 4DVAR itself. The results demonstrate that the 4DVAR technique retrieves wind well for all cases, but is less accurate for temperature retrieval.

The sensitivity of retrieval quality to the initial guess is examined by testing several initial guesses. The results show that retrieval is less sensitive to the first guess. The sensitivity to observational frequency is next examined. To imitate the scanning feature of lidar, different strategies for sampling observational data are practiced. At one extreme, only two volume scan datasets are provided to the model. The accuracy of retrieved velocity and temperature data measured by the correlation coefficient and rms error is still quite satisfactory. With increasing observational frequency, the accuracy of retrieved temperature data is notably improved. Applying an adaptive scanning strategy might improve retrieval quality at desired locations by increasing regional observational frequency.

The sensitivity of retrieval quality to data errors of various amplitude and spatial correlation is also investigated. The study finds that retrieval without temporal and spatial smoothness penalty constraints is less sensitive to horizontally correlated error. This may be attributed to diverse horizontal length scales of turbulent eddies as compared with that of the correlated error. Smoothness penalty constraints significantly improve retrieval quality. The retrieved velocity vector fields with smoothness constraints become less sensitive to spatially correlated error.

Implementation of the surface flux model improves the accuracy of retrieved temperature, especially in the lower mixed layer. Use of the Dirichlet boundary condition at the surface for horizontal velocity components and temperature yields better results than does the Neumann boundary condition.

Doubling eddy diffusivity improves retrieval quality. The diffusive term may act as an effective spatial smoothness function, exchanging information on spatial correlation of data and improving retrieval quality when data are lacking. Height-dependent eddy viscosity and diffusivity retrieve data of quality comparable to constant ones, implying that retrieval is driven more by the observations than by the subgrid-scale model within the assimilation time window.

The spatial smoothness penalty constants for the retrieval of microscale structures are much smaller than those previously used for the mesoscale cases (Sun and Crook 1994). This may be due to the large number of grid points used in the retrieval of microscale flow structures and the large second-order derivative associated with microscale turbulent eddies. The smoothness penalty constraints greatly improve the retrieval of microscale structures and will be applied to real data retrieval.

In summary, the existing 4DVAR technique has been enhanced and extended to accurately retrieve microscale turbulent eddy structures in a simulated CBL. Assimilation of two volume scan radial velocity data spanning 5 min into the 4DVAR model can recover sufficiently complete 3D velocity and temperature data. Various bottom boundary conditions are evaluated and smoothness constants for optimal retrieval are recommended. The adverse effect of spatially correlated errors on retrieval quality is reduced by imposing smoothness penalty constraints. The controlled increase of eddy diffusivity can improve retrieval quality. These conclusions are based on identical twin experiments. Application of this technique to real data requires further investigation.

## Acknowledgments

This work is supported by the National Science Foundation through Grant ATM-9874925, monitored by Dr. Roddy R. Rogers, and by the Iowa Space Grant Consortium. The authors thank Bill Eichinger and Jack Glendening for their review of this document, and Laurent Jay for valuable discussions. Acknowledgment is also made to the National Center for Atmospheric Research, the National Partnership for Advanced Computational Infrastructure, and the National Center for Supercomputing Applications, all sponsored by the National Science Foundation, for computing time used in the research.

## REFERENCES

Cooper, D. I., , W. E. Eichinger, , R. E. Ecke, , J. C. Y. Kao, , J. M. Reisner, , and L. L. Tellier, 1997: Initial investigations of microscale cellular convection in an equatorial marine atmospheric boundary layer revealed by lidar.

,*Geophys. Res. Lett***24****,**45–48.Ferziger, J. H., 1993: Subgrid-scale modeling.

*Large Eddy Simulation of Complex Engineering and Geophysical Flows,*B. Galperin and S. A. Orszag, Eds., Cambridge University Press, 37–54.Grund, C. J., and Coauthors. . 1997: The high resolution Doppler lidar: A new tool for boundary layer research. Preprints,

*12th Symp. on Boundary Layers and Turbulence,*Vancouver, BC, Canada, Amer. Meteor. Soc., 15–16.Hagelberg, C. R., , D. I. Cooper, , C. L. Winter, , and W. E. Eichinger, 1998: Scale properties of microscale convection in the marine surface layer.

,*J. Geophys. Res.—Atmos***103****,**16897–16907.Kalnay, E., , S. K. Park, , Z-X. Pu, , and J. Gao, 2000: Application of quasi-inverse method to data assimilation.

,*Mon. Wea. Rev***128****,**864–875.Lenschow, D. H., , J. C. Wyngaard, , and W. T. Pennell, 1980: Mean-field and second-moment budgets in a baroclinic, convective boundary layer.

,*J. Atmos. Sci***37****,**1313–1326.Lin, C-L., 1999: Near-grid-scale energy transfer and coherent structures in the convective planetary boundary layer.

,*Phys. Fluids***11****,**3482–3494.Liu, D. C., , and J. Nocedal, 1989: On the limited-memory BFGS method for large scale optimization.

,*Math. Programming***45****,**503–528.Long, R. B., , and W. C. Thacker, 1989: Data assimilation into a numerical equatorial ocean model. Part 2: Assimilation experiments.

,*Dyn. Atmos. Oceans***13****,**413–439.Moeng, C-H., 1984: A large-eddy-simulation model for the study of planetary boundary-layer turbulence.

,*J. Atmos. Sci***41****,**2052–2062.Rabier, F., , and P. Courtier, 1992: Four-dimensional assimilation in the presence of baroclinic instability.

,*Quart. J. Roy. Meteor. Soc***118****,**649–672.Roberts, R., , C. Mueller, , J. Wilson, , A. Crook, , J. Sun, , and S. Henry, 1998: Operational application and use of NCAR's thunderstorm nowcasting system. Preprints,

*15th Int. Conf. on Interactive Information and Processing Systems for Meteorology, Oceanography, and Hydrology,*Dallas, TX, Amer. Meteor. Soc., 158–161.Sullivan, P. P., , J. C. McWilliams, , and C-H. Moeng, 1994: A subgrid-scale model for large-eddy simulation of planetary boundary-layer flows.

,*Bound.-Layer Meteor***71****,**247–276.Sun, J., , and A. Crook, 1994: Wind and thermodynamic retrieval from single-Doppler measurements of a gust front observed during Phoenix II.

,*Mon. Wea. Rev***122****,**1075–1091.——, and ——,. 1996: Comparison of thermodynamic retrieval by the adjoint method with the traditional retrieval method.

,*Mon. Wea. Rev***124****,**308–324.——, and ——,. 1997: Dynamical and microphysical retrieval from Doppler radar observations using a cloud model and its adjoint. Part I: Model development and simulated data experiments.

,*J. Atmos. Sci***54****,**1642–1661.——, and ——,. 1998: Dynamical and microphysical retrieval from Doppler radar observations using a cloud model and its adjoint. Part II: Retrieval experiments of an observed Florida convective storm.

,*J. Atmos. Sci***55****,**835–852.——, Flicker, D. W., , and D. K. Lilly, 1991: Recovery of three-dimensional wind and temperature fields from simulated single-Doppler radar data.

,*J. Atmos. Sci***48****,**876–890.Talagrand, O., , and P. Courtier, 1987: Variational assimilation of meteorological observations with the adjoint vorticial equation. I: Theory.

,*Quart. J. Roy. Meteor. Soc***113****,**1311–1328.Thacker, W. C., , and R. B. Long, 1988: Fitting dynamics to data.

,*J. Geophys. Res***93****,**1227–1240.Tuttle, J. D., , and G. B. Foote, 1990: Determination of the boundary layer airflow from a single Doppler radar.

,*J. Atmos. Oceanic Technol***7****,**218–232.Wang, Z., , K. K. Droegemeier, , L. White, , and I. M. Navon, 1997: Application of a new adjoint Newton algorithm to the 3D ARPS storm-scale model using simulated data.

,*Mon. Wea. Rev***125****,**2460–2478.Weckwerth, T. M., , C. J. Grund, , and S. D. Mayor, 1997: Linearly-organized coherent structures in the surface layer. Preprints,

*12th Symp. on Boundary Layers and Turbulence,*Vancouver, BC, Canada, Amer. Meteor. Soc., 22–23.Zou, X., , I. M. Navon, , and J. G. Sela, 1993: Variational data assimilation with moist threshold processes using the NMC spectral model.

,*Tellus***45A****,**370–387.

Descriptions of the retrieval experiments: “obs,” instantaneous 3D radial velocity field; “smooth.,” smoothness penaly constraint; “*u*_{error},” percentage of random error added to initial guess; “*u*_{scale},” the scaling factor for initial guess; “{+}ϵ{-},” observational error

Correlation coefficients and rms errors averaged throughout the boundary layer for different cases at 50 iterations at the middle of the assimilation time window. Units: {+}ϵ{-}_{U}, {+}ϵ{-}_{v}, {+}ϵ{-}_{W} (m s^{−1}) and {+}ϵ{-}_{{+}θ{-}} (K)