Effects of Flow Dependency Introduced by Background Error in Frequent and Dense Assimilation of Radial Winds Using Observation Error Correlated in Time and Space

Tadashi Fujita aMeteorological Research Institute, Japan Meteorological Agency, Tsukuba, Japan

Search for other papers by Tadashi Fujita in
Current site
Google Scholar
PubMed
Close
,
Hiromu Seko aMeteorological Research Institute, Japan Meteorological Agency, Tsukuba, Japan

Search for other papers by Hiromu Seko in
Current site
Google Scholar
PubMed
Close
, and
Takuya Kawabata aMeteorological Research Institute, Japan Meteorological Agency, Tsukuba, Japan

Search for other papers by Takuya Kawabata in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

We investigated the effect of flow dependency in the assimilation of high-density, high-frequency observations. Radial winds from a Doppler radar are assimilated using a regional hybrid four-dimensional variational data assimilation (4D-Var) scheme with a flow-dependent background error covariance. To consistently assimilate 5 km × 5.625° cell-averaged radial winds at an interval of 10 min, the spatial and temporal correlations of the observation error are statistically diagnosed to be incorporated into the hybrid 4D-Var. The spatial correlation width is larger than that expected from instrument error, suggesting a contribution from representation error whose propagation is also considered to lead to temporal correlation, the width of which is diagnosed to increase with forecast time. The background error covariance also has an important role in incorporating observational information into the analysis. Single observation experiments show that the hybrid 4D-Var has more small-scale structure in its flow-dependent background error correlation than the 4D-Var limited from the climatological background error covariance mainly in the former part of the assimilation window. This suggests the higher potential of the hybrid 4D-Var to allow more higher-wavenumber components in the increment. A case study shows that the hybrid 4D-Var makes better use of the dense and frequent observations, reflecting more detailed representation of flow throughout the assimilation window, leading to promising results in the forecast. Sensitivity experiments also show that it is important to use the optimal observation error correlation. It is suggested that the flow-dependent background error becomes necessary to effectively use high-resolution, high-frequency observations.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Tadashi Fujita, tfujita@mri-jma.go.jp

Abstract

We investigated the effect of flow dependency in the assimilation of high-density, high-frequency observations. Radial winds from a Doppler radar are assimilated using a regional hybrid four-dimensional variational data assimilation (4D-Var) scheme with a flow-dependent background error covariance. To consistently assimilate 5 km × 5.625° cell-averaged radial winds at an interval of 10 min, the spatial and temporal correlations of the observation error are statistically diagnosed to be incorporated into the hybrid 4D-Var. The spatial correlation width is larger than that expected from instrument error, suggesting a contribution from representation error whose propagation is also considered to lead to temporal correlation, the width of which is diagnosed to increase with forecast time. The background error covariance also has an important role in incorporating observational information into the analysis. Single observation experiments show that the hybrid 4D-Var has more small-scale structure in its flow-dependent background error correlation than the 4D-Var limited from the climatological background error covariance mainly in the former part of the assimilation window. This suggests the higher potential of the hybrid 4D-Var to allow more higher-wavenumber components in the increment. A case study shows that the hybrid 4D-Var makes better use of the dense and frequent observations, reflecting more detailed representation of flow throughout the assimilation window, leading to promising results in the forecast. Sensitivity experiments also show that it is important to use the optimal observation error correlation. It is suggested that the flow-dependent background error becomes necessary to effectively use high-resolution, high-frequency observations.

© 2022 American Meteorological Society. For information regarding reuse of this content and general copyright information, consult the AMS Copyright Policy (www.ametsoc.org/PUBSReuseLicenses).

Corresponding author: Tadashi Fujita, tfujita@mri-jma.go.jp

1. Introduction

Rapid progress in remote sensing technologies, including radar meteorology, has brought us increasingly high-resolution, high-frequency observational data. It is important to effectively assimilate these data to initialize numerical weather prediction (NWP) to enhance accuracy in forecasting high-impact, severe weather events, which often involve small scale phenomena in space and time. To this end, an appropriate handling of observation error correlation is required to allow the use of dense observations in data assimilation (DA). Conventional DA schemes are often based on the assumption that the observation error correlation is negligible. This commonly leads to a thinning of observations, an inflation of observation errors, and an introduction of superobservations, limiting the use of the information contained in the data (Rainwater et al. 2015; Janjić et al. 2018; Hólm et al. 2018; Fowler 2019; Bédard and Buehner 2020). Working on this important issue, studies have been conducted on the roles of the observation error correlation in DA with idealized cases (Miyoshi et al. 2013; Stewart et al. 2013; Terasaki and Miyoshi 2014; Rainwater et al. 2015; Fowler 2019; Bédard and Buehner 2020). Investigations with real data have mainly addressed the interchannel correlation of observation error of satellite data (Bormann and Bauer 2010; Bormann et al. 2010; Stewart et al. 2014; Waller et al. 2016a), showing improvement of forecast accuracy by accounting for the correlations in the DA (Weston et al. 2014; Bormann et al. 2016; Campbell et al. 2017). Spatial correlation of observation error has been diagnosed for densely distributed radial winds from Doppler radars using three-dimensional variational (3D-Var; Wattrelot et al. 2012; Waller et al. 2016c) and local ensemble transform Kalman filter (LETKF; Waller et al. 2019; Zeng et al. 2021) DA systems. Based on their diagnosis, Simonin et al. (2019) showed a neutral to positive impact on forecast skill by introducing the correlated observation error into the operational 3D-Var to allow the use of dense radial wind data.

Handling of time-correlated observation errors is a common issue in various fields where models are optimized using time series of measurements (e.g., Bryson and Henrikson 1968; Pinnington et al. 2016; Evensen and Eikrem 2018). As for DA for NWP, the ECMWF introduced time correlation of observation errors when assimilating time sequences of hourly surface pressure observations using the four-dimensional variational (4D-Var) scheme (Järvinen et al. 1999). With the recent sophistication of remote sensing technologies, the consideration of the time correlation of observation error has become increasingly important toward the use of high-frequency observations in four-dimensional DA. In view of this, Bennitt et al. (2017) recently provided the diagnosis of the temporal error correlation for observations from ground-based global navigation satellite system. The present study includes the diagnosis of the time correlation of radial wind observation error required to assimilate radial winds at high frequency in the 4D-Var used in the former operational Mesoscale Analysis (MA; JMA 2019) operated at Japan Meteorological Agency (JMA) until 2020.

Once observations are input into the DA scheme at high resolution in time and space, it is essential to handle the flow-dependent features at the appropriate resolution. The flow dependency of the background error is partly accounted for through the propagation of perturbations in the 4D-Var. However, the climatological background error covariance matrix (denoted by B) at the beginning of the assimilation window is often drastically simplified, possibly limiting the atmospheric flow represented in increments. Recent advanced DA widely uses an ensemble-based flow-dependent B, including DA at a convective scale that deals with severe events highly dependent on atmospheric states localized in time and space (Gustafsson et al. 2018).

The present study uses radial winds, densely distributed in time and space, to explore their effective use in a hybrid 4D-Var with flow-dependent B. An extended diagnosis of observation error is performed with both temporal and spatial correlations to be incorporated into the hybrid 4D-Var. And we investigate how the use of flow-dependent B improves the extraction of information from dense, frequent observations in comparison with the 4D-Var.

Section 2 describes the handling of radial winds in MA. In section 3, observation error correlation profiles of radial winds are diagnosed. In section 4, impacts of including observation error correlation are investigated using a simple variational scheme. Section 5 shows results of assimilating radial winds densely distributed in time and space by the hybrid 4D-Var and the 4D-Var with the correlated observation error. Section 6 gives the discussion and conclusion.

2. Radial wind data in MA

The present study uses an experimental system based on MA (JMA 2019) used by JMA until 2020, adapting the 4D-Var of JMA nonhydrostatic model-based variational data assimilation (JNoVA; Honda et al. 2005), whose adjoint model is used in a cloud-resolving 4D-Var for research purposes (Kawabata et al. 2007, 2011). MA runs a 3-hourly 4D-Var cycle with an assimilation window of 3 h to initialize the 5-km operational limited-area model called the Mesoscale Model (MSM; JMA 2019). The model domain is Japan and its surrounding areas (4080 km × 3300 km). The incremental approach (Courtier et al. 1994) is applied, running the inner model in the iterative optimization of the 4D-Var at a lower horizontal resolution of 15 km to calculate the increments with reduced computational resources, while the outer model at a higher horizontal resolution of 5 km is used to generate the first guess and the analysis. The inner and outer models have 38 and 48 vertical layers, respectively, with the model top at ∼22 km. MA assimilates conventional observations and various satellite and ground-based remote sensing observations, including radial winds from Doppler radars. A detailed description on the configuration of MA is found in JMA (2019).

The quality control (QC) processes and the observation operator of radial winds in MA (Ishikawa and Koizumi 2006; Seko et al. 2004; JMA 2019) are briefly described in the appendix A. The radial winds are averaged over 5 km (in beam range) × 5.625° (in azimuthal angle) cells for use in the MA. The observation errors of radial winds, along with those of other observations, are assumed to be uncorrelated with each other in the MA. To meet this assumption, the cell-averaged radial winds are further thinned to 20 km on each elevation angle to be assimilated in the 4D-Var hourly on the hour. The standard deviation of observation error applied in the MA is set to be equal to 3.0 m s−1 or to the standard deviation of the cell samples, whichever is larger. In practice, the value of 3.0 m s−1 is adopted in most cases. The Sapporo radar is one of the sources of radial winds located at 43.14°N, 141.01°E (Fig. 2). Profiles of beam heights at different elevation angles as a function of the distance from the Sapporo radar (Fig. 1) were determined according to Doviak and Zrnić (1993) and used in the MA.

Fig. 1.
Fig. 1.

The beam altitude used in MA against distance from the Sapporo radar site, for the elevation angles 0.1° (red), 1.1° (green), 2.6° (blue), and 4.3° (black).

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

3. Statistical diagnosis of the radial wind observation error

a. Diagnostic procedure

In this section, we followed Desroziers et al. (2005) to diagnose the observation error covariance R for radial winds. This methodology estimates R from statistics of observation-minus-background and observation-minus-analysis residuals. It has been applied to estimate the spatial observation error correlation of radial winds on operational 3D-Var and LETKF DA systems (Wattrelot et al. 2012; Waller et al. 2016c, 2019; Zeng et al. 2021). In the present study, the temporal correlation is also estimated for use in the 4D-Var.

A 3-hourly 4D-Var DA cycle based on MA was run from 0000 UTC 1 July to 0000 UTC 8 July 2018 to generate statistical samples. During this period of radial wind data collection from the Sapporo radar, there was continuous precipitation due to a stationary front and a low pressure system around this region (Fig. 2). Both the DA cycle and the diagnosis use the cell-averaged observations, unless otherwise noted.

Fig. 2.
Fig. 2.

A 9-h forecast of 3-h accumulated precipitation [mm (3 h)−1; shaded] and sea level pressure (hPa; contoured every 1 hPa) valid at 1200 UTC 3 Jul 2018. Locations of the Sapporo and Kushiro radars and the range of assimilated radial winds (150 km from the radar site) are indicated with black pluses and circles, respectively.

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

MA assimilates radial winds thinned to 20 km hourly using a diagonal R without correlation. However, the diagnosis derives a non-diagonal R, using observations without spatial thinning and at a time interval of 10 min to determine the detailed structure of the error correlation. Due to these treatments, the analysis can become suboptimal and affect the diagnosis of R. Therefore, to improve the consistency of the diagnosis, a second 4D-Var cycle is performed to re-diagnose R, assimilating radial winds from the Sapporo radar without spatial thinning and at a time interval of 10 min, and applying a nondiagonal R based on the first diagnosis as modeled in section 3c below. The standard deviation of R is taken to be the same with that of the MA to focus on the effect from introducing the observation error correlation. Radial winds from sites other than Sapporo and other observations are assimilated as in the MA. The second diagnosis of R shows an increase in the standard deviation and the correlation width compared to the first ones, but the difference is not significant compared to deviations from the modeling of R (section 3c) used in the assimilation. Thus, the results from the second diagnosis are first presented in sections 3b and 3c, and the difference between the diagnoses along with additional sensitivity experiments are discussed in section 3d below.

b. Diagnosed profile of observation error covariance

The standard deviation (Fig. 3a) varies mostly within 1.5–2.5 m s−1, except for high altitudes with decreasing statistical samples (Fig. 3b). It is comparable to 1.95 m s−1 from Waller et al. (2016c) and is reasonable compared to those from Waller et al. (2019), Wattrelot et al. (2012), and Zeng et al. (2021) considering differences in experimental settings. This is below the standard deviation of observation error used in the MA, set to be 3 m s−1 in most cases. Profiles of the error from different elevation angles look more similar as a function of the altitude (Fig. 3b) than as a function of the beam range (not shown). This suggests that height dependent atmospheric conditions have a large contribution to determine the profile of the observation error. The standard deviation is larger at the lowest altitudes (below 2000 m), likely to be affected by various sources of signal noise near the surface. Except for this, it overall tends to increase with altitude (2000–4000 m) as wind speed increases. These profiles are similar to Waller et al. (2016c, 2019). It is noted that all the elevation angles reaching to 4500–5000 m show an increase around these altitudes, where the melting layer is located in this case. Waller et al. (2016c) reported that the error from radial wind observation operator can be reduced by accounting for the effect of intense reflectivity from a bright band. The same situation is possible, since this effect is not considered in the observation operator (appendix A).

Fig. 3.
Fig. 3.

(a) Standard deviation of the radial wind observation error from the Sapporo radar, plotted against the altitude for elevation angles 0.1° (red), 1.1° (green), 2.6° (blue), and 4.3° (black) from the second diagnosis, for elevation angle 2.6° from the first diagnosis (blue dotted line), from the diagnosis using the flow-dependent B (dark blue dotted line), and from the diagnosis using the raw data (light blue dotted line). The statistical samples are taken over the period of 0000 UTC 1 Jul 2018–0000 UTC 8 Jul 2018. (b) Number of observations used in the statistics in the second diagnosis.

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

Figure 4 shows the diagnosed observation error correlations. It is noted that the correlation from the statistics itself is not symmetrical because of the estimation procedure. The correlation along the beam at the elevation angle of 1.1° (Fig. 4a) has a half-width (a half-width at half-maximum) of ∼10–25 km. This width is consistent with ∼10 km obtained by Waller et al. (2016c) using the method by Desroziers et al. (2005) for an elevation angle of 1° and a beam range of ∼94 km, considering the many differences in configuration. It is, however, considerably larger than that obtained by Xu et al. (2007a,b) focusing on the instrument error, less than 1 km. The representation error (e.g., Janjić et al. 2018) is considered to have a large contribution to the observation error correlation obtained in this study. The original radial wind data are measured at high resolution, and the cell-averaged values are considered to be representative of a resolution of the cell scale, 5 km × 5.625°. On the other hand, the phenomena that can be resolved by the DA systems are several times larger than the grid spacing, 5 km for the outer model, while it is 15 km for the inner model. Furthermore, the 4D-Var cycle uses the climatological B, which also limits the resolution. This can lead to errors correlated at the scale of phenomena with discrepancies in the representations between the observations and the DA system. It is noted that the first guess is not used in the cell-averaged observations (appendix A), and this does not contribute to the diagnosed correlation. The correlation width increases with the beam range, which is also consistent with the findings of Waller et al. (2016c), and is likely to result from larger measurement volumes at far ranges. Although the observation operator considers vertical beamwidth, it does not take into account that the size of the averaging cell increases its scale along the beam (appendix A), which can induce correlated errors along the beam. Errors concerning larger scale phenomena at higher altitudes also can increase the correlation at far ranges. However, the increase of the width with the beam range can only be seen at lower elevation angles of 0.1° and 1.1°, but it is not seen, and the correlation width itself is smaller, at higher-elevation angles of 2.6° and 4.3° (not shown). It is inferred that a large gradient along the beam path (Fig. 1), increasing with the beam range, can work oppositely to reduce the correlation width along the beam, reflecting a smaller correlation scale in the vertical direction. Waller et al. (2016c) also shows that the correlation width along the beam decreases as the elevation angle increases.

Fig. 4.
Fig. 4.

Observation error correlation of radial winds from the Sapporo Doppler radar. Statistic period is 0000 UTC 1 Jul–0000 UTC 8 Jul 2018. (a) Correlation along the beam at elevation angle of 1.1°. The statistical samples are taken from all the azimuthal angles and times. The width of the cell is 5 km. Data at short ranges are rejected by QC (appendix A). (b) Time correlation at the elevation angle of 1.1°. Samples are taken from all the azimuthal angles and beam ranges. The width of the cell is 10 min. Data at the beginning (0 min) of the 3-h assimilation window is not used, because they are already used at the end (180 min) of the assimilation window in the previous analysis of the 3-h cycle. (c) Correlation in azimuthal angle at the elevation angle of 1.1°. Samples are taken from all the times. The size of the cell is 5 km × 5.625° in beam range and azimuthal angle directions, respectively. (d) Correlation in elevation angle. Correlations between the error at the elevation angle of 0.1° (red), 1.1° (green), 2.6° (blue), and 4.3° (black) and those at other elevation angles. The statistical samples are from all the beam ranges, azimuthal angles, and times. Black contours in (a)–(c) indicate correlation values of 0.2 and 0.5. The gray contours in (a)–(c) indicate the number of pairs contoured every 4000.

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

Figure 4b shows the diagnosed time correlation of the observation error at the elevation angle of 1.1°. The statistics shows a half-width of ∼30–60 min, increasing with the forecast time, suggesting a contribution from the error involved with the forecast model. Differences between model and observation representations due to a limited resolution and use of parameterizations in the DA system (Janjić et al. 2018) can propagate to cause temporal correlation in the representation error. It is noted that the diagnosis in this study does not take into account the model error, which may also contribute to the time correlation of the error.

The statistics gives the half correlation width of ∼15 km in the azimuthal direction (Fig. 4c). It is similar to that obtained in Waller et al. (2016c), for the elevation angle of 1°, and is larger than those obtained for the elevation angle of 1.5° in Waller et al. (2019) and Zeng et al. (2021), ∼4–10 km. The width again shows a slight increase with the beam range, as the averaging cell size and the beamwidth increase, varying the representative scale of observations. This increase is consistent with the results from Waller et al. (2016c) and Zeng et al. (2021). The correlation in the azimuthal direction, taken horizontally, shows lower sensitivity to the elevation angles compared to the correlation along the beam (not shown). At short ranges around 180° azimuth, where statistical sample increases, there is a negative correlation that may correspond to the representation error for winds crossing the radar site.

On the contrary, the statistical correlations between observation errors at different elevation angles are small, taking values near or smaller than 0.3 (Fig. 4d). The statistics are taken for pairs with equal beam range, azimuthal angle, and time, but different elevation angles. Large intervals of the sampled elevation angles, 0.1°, 1.1°, 2.6°, and 4.3°, as compared to the beamwidth used in the observation operator, 0.3°, may contribute to reduction in the correlations.

Figure 5 (filled circles) displays the diagnosed observation error correlations from the statistics of samples from all the elevation angles, beam ranges, azimuthal angles, and times. The correlation along the beam (Fig. 5a) rapidly decreases from 1 with the beam-range separation, reaching to 0.3 at ∼25 km. It shows a bump-shaped structure with a size of ∼0.1–0.2 extending to ∼100 km before approaching zero. The time correlation (Fig. 5b) decreases from 1 with time interval, but also has a bump-shaped structure with a larger size of ∼0.25 reaching the full range of the assimilation window. We did not investigate whether these bump-shaped structures indicate the existence of biases or just noise originated from a limited number of statistical samples and a limited statistical period. We found that the lowest elevation angle of 0.1° gives a large beam-range correlation width at middle beam ranges, which is likely to have large contributions from a limited number of cases. The QC process, such as rejecting cell-averaged radial winds less than 5 m s−1, which has been introduced into the operational MA based on past monitoring to avoid noise found in the original data (see appendix A), can also have an influence on the statistical error profile especially at lower elevation angles with small wind speed. Further investigations are needed to obtain a more robust estimation of the observation error profiles. The correlation in the azimuthal direction (Fig. 5c) is obtained by taking statistics of pairs from all the beam ranges binned with an azimuthal distance of 1 km, accounting for pairs of observations with an equal beam range, elevation angle, and time. The correlation drops from 1 to 0.8 over a very short distance. After that, the correlation decreases with distance, reaching 0.3 at ∼25 km, and approaches zero over larger distances without a bump-shaped structure. Thus, the correlation in azimuthal direction, taken between samples from different beams, shows a different profile in its tail compared to the correlation within the beam.

Fig. 5.
Fig. 5.

Correlation profiles from the diagnosis. The statistical period is 0000 UTC 1 Jul–0000 UTC 8 Jul 2018. (a) Correlation along the beam (sampled from all the times, azimuthal angles, and elevation angles). (b) Time correlation (sampled from all the beam ranges, azimuthal angles, and elevation angles). (c) Correlation in azimuthal direction (sampled from all the time, beam ranges, and elevation angles). Results from the second diagnosis (black filled circles), the first diagnosis (open circles), the diagnosis with the flow-dependent B (blue pluses), and the diagnosis with raw observations (red pluses) are shown. Correlation profiles modeled with a Gaussian form for use in the assimilation experiments are plotted with black solid curves. Gray dashed curves indicate the Gaussian-shaped correlation profiles used in the sensitivity experiment in section 5f, with widths of 0.7 times. The numbers of pairs are plotted with gray bars for the second diagnosis (right axis).

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

The correlation along the beam (Fig. 5a) becomes insignificant (<0.2; Liu and Rabier 2002; Waller et al. 2016c) at ∼40 km, which is larger than ∼15 km obtained in Waller et al. (2016c) (Wattrelot et al. (2012) gives an even smaller value). In addition to the bump-shaped structure mentioned above, our larger beam range and lower elevation angles can contribute to the larger correlation width. Our larger grid spacing, 5 and 15 km, compared to theirs, 3 km, also can lead to representation error at larger scales.

The correlation in the azimuthal direction reaches to 0.2 at ∼30 km. This result on average is larger than the horizontal correlations from Waller et al. (2016c, 2019) and Zeng et al. (2021) by ∼5–10 km. It is reasonable considering our low resolution. It is shown that the climatological B used in the 4D-Var cycle also works to increase the correlation width in section 3d.

c. Modeling of observation error covariance for use in DA

Based on the statistics described above, we modeled the observation error correlation to use it in the DA. Correlation is considered in beam range, azimuthal angle, and time directions, while it is neglected in elevation angle direction. The correlation is assumed to follow a Gaussian form as exp[−0.5(Δ/σ)2], where the Δ indicates a separation between observations, and the σ is the correlation scale approximated referring to the statistics described in section 3b. The σ in the beam range, time, and azimuthal directions are taken to be 15 km, 45 min, and 15 km, respectively. The widths are set to be constant for simplicity, focusing on the effect of introducing correlation itself into R used in the DA. The modeled correlation profiles are shown in Fig. 5 with solid curves. The profiles give a rough approximation, showing correlations higher than the diagnoses at short ranges and neglecting the bump-shaped tails, but mostly capture the range of correlations exceeding 0.4.

Computation of R1 is required in variational DA. Since observations are not always present in all cells, and locations of available observations vary for each analysis during the assimilation cycle, R1 is needed to be computed every time reflecting the pair of cells where the observations exist. On the contrary, R1 is constant during a variational optimization in the analysis, and can be calculated in advance of each analysis. The present study applies the cell-averaging to the observations and neglects the observation error correlation between different elevation angles, reducing the computational cost. Thus, the R is inverted for each elevation angle. The size of the matrix varies depending on cases, taking values of ∼5000 on average during the frontal rain in the present experimental period for the observational interval of 10 min, but reaches up to ∼13 000 at the maximum.

Full calculation of the inverse of the Gaussian covariance matrix results in severe spurious noise from the eigenmodes with small eigenvalues. To avoid this, modes with larger eigenvalues are used, up to the mode with accumulated sum of eigenvalues reaching 99% of the full trace. An exponential form as in Simonin et al. (2019) may result in a better fit to the statistical correlation profiles (Fig. 5) with a steep profile at small distances and a long tail. In a trial with the exponential form, the R1 calculation was able to include all the eigenmodes, while it resulted in a smaller impact on the forecasts in the case investigated in section 5. Thus, the following part uses a Gaussian form. An appropriate model for the observation error correlation will be investigated in future work. The effect from the truncation of eigenmodes is discussed in appendix B.

d. Sensitivity experiments of the diagnosis

The method by Desroziers et al. (2005) assumes that the analysis update determined by B and R used in the DA cycle is consistent with the true one (Desroziers et al. 2005; Wattrelot et al. 2012; Waller et al. 2016b). This is not necessarily true in real DA systems and may affect the diagnosis of the observation error.

As in section 3a, the diagnoses are performed twice in this study to improve consistency. The observation error standard deviation at the elevation angle 1.1° (Fig. 3a) thus obtained is larger in the second diagnosis (blue solid line) than in the first one (blue dotted line). Figures 5a–c show correlation widths are also larger in the second diagnosis (filled circles) than in the first one (open circles). Waller et al. (2016b) showed, from theoretical considerations, both diagnosed observation error standard deviation and correlation width were larger for DA with observation error correlation than for DA neglecting it. The results are consistent with this. However, the first diagnosis may also have been affected by the different radial wind thinning in the DA cycle and in the diagnosis.

To make the error covariances used in the DA cycle closer to the correct ones, we conduct another diagnosis using the nondiagonal R for radial winds from the Sapporo radar as in the second one, but using the flow-dependent B (section 5). The flow-dependent B is generated by an ensemble of data assimilations (EDA; Isaksen et al. 2010) with the same configuration as in section 5a, but run separately using the regenerated perturbed observations. The DA cycle is conducted from 0000 UTC 1 July to 0000 UTC 8 July 2018 as in the previous diagnoses, but the flow-dependent B is used starting from the 0300 UTC 2 July analysis. The hybrid 4D-Var in the DA cycle uses ensemble perturbations from the EDA, but the EDA runs independently of the hybrid 4D-Var. Both the standard deviation (dark blue dotted line in Fig. 3a) and the correlation width (blue pluses in Figs. 5a–c) decrease compared with those from the second diagnosis (blue solid line in Fig. 3a and black circles in Figs. 5a–c). The azimuthal correlation (Fig. 5c) now reaches to 0.2 at ∼22 km, which is more consistent with Waller et al. (2016c) and Zeng et al. (2021). In the period of this experiment, the variance of the flow-dependent background error is expected to be larger than that of the climatological one due to the stationary front near Sapporo (section 5b), although the two are comparable in the domain average. On the other hand, near disturbances associated with the front, the correlation distance is expected to be smaller for flow-dependent background error than for climatological one (section 5b). According to Waller et al. (2016b), as the background error variance assumed in the DA increases, the diagnosed observation error variance and correlation width decrease. While as the background error correlation width assumed in the DA decreases, the diagnosed observation error correlation width increases. And the effect due to the former is stronger. The results are consistent with this. In addition, the effect from the change in the first guess, and the true background error, through the cycling is particularly large when B used in the DA is replaced. Because the flow-dependent B can represent phenomena down to smaller scales than the climatological B (section 5b), the representation error is expected to reduce at larger scales, which will also contribute to decrease the observation error variance and correlation width.

In this study, both the DA cycle and the diagnosis uses cell-averaged observations, where raw observations with a resolution of 250 m × 0.703° are averaged over a 5 km × 5.625° cell. To investigate the effect of the cell averaging, an experiment was conducted to use the raw observation closest to the cell center in the DA cycle and the diagnosis. A raw observation is only used if it exists within 500 m × 0.703° from the cell center, applying the QC (appendix A), and is treated as being at the cell center. As in the second diagnosis, the DA cycle uses the nondiagonal R based on the first diagnosis and the climatological B. The diagnosed observation error standard deviation (light blue dotted line in Fig. 3a) is larger than that from the second diagnosis (blue solid line in Fig. 3a). This is considered to be due to the contribution from instrument errors, as well as that from representation errors at scales below the cell size. The observation error correlation from the diagnosis (red pluses in Figs. 5a–c) shows contributions from errors with very short correlation widths, indicating their random nature. Especially, Fig. 5c shows that the components contributing to the correlation values from 0.65 to 1.0 have width less than 2 km, closer to that obtained in Xu et al. (2007a,b). These results suggest that the cell averaging works to reduce the relative proportion of errors with smaller scales, resulting in an increase in the observation error correlation width. The azimuthal correlation (red pluses in Fig. 5c) reaches to 0.2 at ∼25 km, compared to ∼30 km from the second diagnosis (filled circles). Waller et al. (2016c) showed that the horizontal correlation length scale was slightly reduced when raw observations were used instead of super observations. The results in this study are consistent with this, but show a larger effect. This is reasonable considering that the averaging cell contains 160 raw data, over 10 times more than 15 in Waller et al. (2016c).

4. Effect of correlated observation error in variational DA

The effects of observation error correlations on DA are investigated using a simple variational scheme. This section runs this scheme in two dimensions, beam range and azimuthal angle, but here we describe a more extended version that includes also the time dimension, which will be used in section 5c. Thus, the variational DA runs on a 2 + 1-dimensional space with 30 × 64 × 19 grid points at a resolution of 5 km × 5.625° × 10 min. Innovations of radial winds are located on the DA grid points. For simplicity, the analysis variable is taken to be radial wind itself, and the observation operator simply gives the grid point value at the observation point. This scheme includes the time dimension, but does not run the model and does not take time evolution into account. Observation error correlations are set as in section 3c, that is, σ is taken to be 15 km, 15 km, and 45 min in the beam range, azimuthal angle, and time directions, respectively. The background error correlation also assumes a Gaussian form with the same scales as the observation error correlation. Observation and background error variances are taken to be (1 m s1)2. Since the correlation scales and variances of the background and observation errors are the same, the analysis increments are expected to take values roughly half of the innovations.

In this section, we run this scheme on a two-dimensional plane, assimilating innovations of radial winds from the Sapporo radar (Fig. 6a). Accounting for the observation error correlations (Fig. 6b), the increments reflect the detailed spatial distribution of innovations consistently over the whole domain. Neglecting the observation error correlations, the detailed structure is still present (Fig. 6c). However, larger increments dominate where innovations with identical sign are grouped, giving the analysis close to the observations (circles). Thus, it is suggested that accounting for the correlation of the observation error helps to appropriately control increments from densely distributed observations.

Fig. 6.
Fig. 6.

Results from the DA experiments using the simple scheme. Radial wind innovations are taken at the elevation angle of 1.1° from the Sapporo radar valid at 0450 UTC 3 Jul 2018. (a) Innovations. (b) Increments with the full R. (c) Increments neglecting off-diagonal elements of R. (d) Increments neglecting off-diagonal elements of R, but with observation error standard deviation multiplied by 3. The size of the cell is 5 km and 5.625° in beam range and azimuthal angle directions, respectively. Black ellipses in (c) indicate the characteristic parts of the experiment (see text).

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

The radial power spectrum of the increments (Fig. 7) shows that the amplitude of the increments is reduced at all wavenumbers in the case with the observation error correlation (red solid line) compared to the case without it (black solid line). On the other hand, the ratio of the power shows that the case with the correlation has relatively larger ratios for components in wavenumbers 3–6. According to the theoretical consideration of Fowler et al. (2018), when the observation error correlation is considered, the eigenvalues of the low (high) wavenumber modes of the observation error become larger (smaller) than when it is neglected. As a result, the sensitivity of the analysis to observations increases at higher wavenumbers. The small ratios below wavenumber 2 and the large ones at wavenumbers 3–6 in this study are consistent with this. However, the ratio at the even higher wavenumber above 6 is small. This may be due to the truncation of the eigenmodes of the observation error described in section 3c (see appendix B for the effect in a real case), which reduces the constraint on the modes corresponding to these higher wavenumbers.

Fig. 7.
Fig. 7.

Radial power spectra of the analysis increments discussed in section 4 (see Fig. 6). The spectra are obtained by interpolating the increments of 30 grid points (150 km) along the beam to 32 points for each azimuthal angle, applying the discrete Fourier transform, and averaging the squared norm for all the 64 azimuthal angles. Results are shown for the case with the full R (red); the case neglecting off-diagonal elements of R (black); the case neglecting off-diagonal elements of R, but with observation error standard deviation multiplied by 3 (green); and the case neglecting off-diagonal elements of R, but with innovations thinned to one-third in both directions (blue). A solid line indicates the power spectrum. A dotted line indicates the ratio of the power from the experiment against that from the case neglecting off-diagonal elements of R (right axis). The gray solid line indicates the ratio 1.

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

As is often done in DA systems assuming an uncorrelated observation error, an experiment is performed with the diagonal R to inflate the observation error standard deviation, by a factor of 3 (Fig. 6d). The peak values of the increment get closer to the case with the full R (Fig. 6b), but the increment pattern is smoothed out, and the detailed structure is lost. The situation is the same when the observations are thinned down to one-third in the beam range and azimuthal directions (not shown). In the radial power spectra (Fig. 7) of these cases (green and blue dotted lines), the ratios at wavenumber 2 and below are relatively larger than those at wavenumber 4 and above, indicating that the lower wavenumbers are dominant compared to the case with the full R (red dotted line). These results show that using a full R offers a detailed reflection of innovation patterns to the analysis.

5. Hybrid 4D-Var assimilation with observation error correlated in time and space

The results in the previous section suggest that detailed information can be consistently incorporated into analysis by appropriately considering the correlation of observation errors. To improve the forecasts from the analysis, it is also important to assimilate observations in line with the time evolution of a forecast model. Furthermore, the flow dependency represented by a background error is expected to be essential to better extract detailed information from high-density, high-frequency observations depending on various atmospheric conditions. In this section, both temporal and spatial correlations of radial wind observation error are applied to the hybrid 4D-Var, as a DA system dealing with time evolution in its optimization and equipped with a flow-dependent B. The results are discussed and compared with those from the 4D-Var with only a climatological B.

a. Configuration of the experiment

The configurations of the hybrid 4D-Var and the 4D-Var are summarized in Table 1. The observation error correlation scales are specified as in section 3c, roughly based on the statistics. On the contrary, in order to focus on the effect of correlation in the observation error, the observation error standard deviation is set simply as in the operational MA, mostly 3 m s1, not based on, or optimized to the present statistics.

Table 1

Configuration of the hybrid 4D-Var and the 4D-Var in the experiments.

Table 1

The hybrid 4D-Var is implemented by extending the control variables of the JNoVA 4D-Var to include flow dependency from an ensemble of perturbations (Buehner 2005) generated using an EDA (Isaksen et al. 2010). The EDA with six members runs 3-hourly 4D-Var cycles assimilating all the MA observations without observation error correlation at the same resolution as in the MA (Fig. 8). All the observations except for precipitation rate data are perturbed with normal random numbers according to their diagonal R. A 54-member ensemble is formed to estimate the flow-dependent B, using the time-lagged ensemble technique with forecasts from the latest nine initial times of the 6-member EDA (0–24-h forecasts). The domain average of the spread of the ensemble background error has a seasonal variation, but is found to be comparable to the standard deviation of the climatological background error in the present experimental period. In incorporating the flow dependency into the control variable, the ensemble B and the climatological B are weighted as 0.8:0.2. The localization function is set to take a Gaussian form, reaching to e0.5 at scales of 75 km and 10 model layers in horizontal and vertical directions, respectively.

Fig. 8.
Fig. 8.

A schematic diagram of the EDA. The PO indicates perturbed observations.

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

Using the configuration described above, radial wind observations from the Sapporo radar are assimilated to generate an analysis valid at 0600 UTC 3 July 2018. The first guess is generated by a 3-h forecast from 0300 UTC 3 July 2018. A 4D-Var cycle as in MA is run from 0000 UTC 1 July to 0300 UTC 3 July 2018, and the analysis from this cycle is used to initialize the 3-h forecast. The first guess thus obtained is used to estimate the innovations of radial wind over the assimilation window of 0300–0600 UTC 3 July 2018. The EDA cycle is also initiated at 0000 UTC 1 July 2018 to generate ensemble perturbations valid at 0300 UTC 3 July 2018, that is, the initial time of the assimilation window.

Experiments of the analysis valid at 0600 UTC 3 July 2018 are performed with four different configurations. All the experiments assimilate only radial winds from the Sapporo radar observed during the 3-h assimilation window ranging from 0300 to 0600 UTC (except for observations just at 0300 UTC). The cell-averaged radial winds are assimilated without a spatial thinning, accounting for the modeled temporal and spatial correlations of observation error. Focusing on the effect of frequency of the observations, impacts are compared between cases with observation intervals of 60 and 10 min.

  1. 4D-Var (without the flow-dependent B). Radial winds are assimilated every 60 min from 0400 to 0600 UTC 3 July 2018.

  2. As in configuration 1, but radial winds are assimilated every 10 min from 0310 to 0600 UTC 3 July 2018.

  3. Hybrid 4D-Var with the flow-dependent B. Radial winds are assimilated every 60 min.

  4. As in configuration 3, but radial winds are assimilated every 10 min.

The forecasts from the analysis uses the JMA nonhydrostatic model (JMA-NHM; Saito et al. 2006), used in the operational MSM until 2017. This is the same model as the outer model of the JNoVA 4D-Var used in the present study. The extended forecasts from the first guess, generated without assimilating the radial winds from 0300 to 0600 UTC 3 July 2018, are also used for the verification of forecasts from the analysis.

b. Properties of the background error covariance

Prior to the radial wind assimilation experiments, the properties of B are investigated by means of the single-observation experiment. The increment Δx(i,t) at a given grid point i and time step t from a single-observation experiment, where the observation is placed at a grid point io and time step to, is proportional to the background error covariance B(i,t)(io,to) between (i, t) and (io, to). Using the observation minus analysis d(io,to)a and the standard deviation of the observation error σo, Δx(i,t) is expressed as Δx(i,t)=B(i,t)(io,to)d(io,to)a/σo2. Here, for simplicity, we assume that the observation point coincides with a grid point of the DA system, and that the element of the observation and that of the increment are the same. And a penalty term is not considered, which is actually included in the present DA system (JMA 2019).

Experiments are conducted placing a wind x-component innovation at the position of the Sapporo radar (43.14°N, 141.01°E), at an altitude of 600 hPa, and at different time steps. The σo is set to 1 m s−1. The increments for an innovation of 4 m s−1 at time step 1 (−180 min) are very different between the hybrid 4D-Var and the 4D-Var. The 4D-Var increments at time step 1 (Figs. 9a,d) reflect the climatological B (JMA 2019), whose horizontal correlation is modeled to have a Gaussian form with a width (the distance where correlation becomes e−0.5) of 134 km in the x direction and 68 km in the y direction on the 17th model layer for the wind x component. Due to the influence of this initial pattern, a large-scale structure is prominent throughout the assimilation window (Figs. 9b,c). The increment of the hybrid 4D-Var at time step 1 (Fig. 9f) has a pattern along the front extending from the west-southwest to the east near Sapporo (Fig. 2), with a more detailed vertical structure (Fig. 9i). As time progresses, it moves eastward with the addition of local small-scale structures (Figs. 9g,h), reflecting disturbances near the front. According to Fowler et al. (2018), as the background error correlation scale decreases, the sensitivity of the analysis to the observations increases at higher wavenumbers. Thus, the analysis from the hybrid 4D-Var is more likely to reflect the details of the smaller scales than that from the 4D-Var, mainly in the former part of the assimilation window in this case.

Fig. 9.
Fig. 9.

Increments of wind x component from the single observation experiments. Results are shown in the inner model space with a horizontal grid spacing of 15 km. A wind x-component observation is placed at the location of the Sapporo radar (43.14°N, 141.01°E), corresponding to (x, y) = (194.28, 168.42), at the height of 600 hPa with the innovation of 4 m s−1 at the beginning of the assimilation window (time step 1, corresponding to −180 min). (a)–(c) The xy cross section for the 4D-Var on the 17th inner model layer, mostly corresponding to ∼600 hPa at time step 1 (−180 min) in (a), time step 136 (−90 min) in (b), and time step 271 (0 min) in (c). (d) The xz cross section for the 4D-Var at y = 168 at time step 1 (−180 min). (e) The xt cross section for the 4D-Var at y = 168 on the 17th model layer. (f)–(j) As in (a)–(e), but for the hybrid 4D-Var. Pluses indicate the location of the observation. Black dotted line in (a) indicates the location of the xz and xt cross sections. The yellow solid line indicates the observation point (x = 194.28). The black dotted line in (e) and (j) indicates the propagation at 20 m s−1 in the x direction starting from the observation point at time step 1.

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

Note that the width of the localization, 75 km (section 5a), is slightly larger in the y direction but is smaller in the x direction on the 17th model layer compared to that of the climatological background error correlation. In this case study, the scale of the flow-dependent background error correlation is small in the frontal disturbance area, and it is expected that the main part of the correlation structure around the peak is captured. However, the localization in the x direction can limit the extension of the correlation along the front. It is desirable to bring the scale of localization closer to that of the climatological background error correlation, refining the ensemble for more general cases.

Looking at the time evolution at the location of the observation point, the main variation is the decay of the increment peak (yellow solid lines in Figs. 9e,j). At time step 1, the wind near the increment peak is approximately 20 m s−1 in the x direction. Along with this, the peak moves eastward at ∼20 m s−1 due to advection by this flow (black dotted lines). In the hybrid 4D-Var (Fig. 9j), the increments have smaller spatial scales, and less extend to the west (Figs. 9g,h). Therefore, it takes less time for the main part of the increment to pass through the observation point than in the 4D-Var (Fig. 9e), and the decay is faster. On the other hand, a wind region of less than 15 m s−1 passes from the west of Sapporo afterward. Therefore, there is also a component of slow eastward propagation, although its amplitude is smaller (Figs. 9e,j).

From the time evolution of the increments at the observation point (Fig. 10a), the standard deviation (Fig. 10b) and time correlation (Fig. 10c) of the background error are evaluated using the relationship described above. Since the evaluation fluctuates, affected by the convergence of the variational minimization, the average of the cases with innovation of 3, 4, and 5 m s−1 is used for the standard deviation and the covariance. The overall trend of the standard deviation (Fig. 10b) is to increase with time. It is mostly larger in the hybrid 4D-Var (blue) than in the 4D-Var (red) in this case, due to the influence of the front over Sapporo region. Consistent with Figs. 9e and 9j, the correlation (Fig. 10c) with −180 min has a smaller width in the hybrid 4D-Var (dark blue line) than in the 4D-Var (dark red line). The correlations have a long tail corresponding to the slow eastward flow seen in Figs. 9e and 9j. The width increases as the correlation is taken with the later time steps (lines with lighter color), partly due to the slow propagation of the corresponding increments (not shown).

Fig. 10.
Fig. 10.

The increments and the estimated background error profiles from the single observation experiments. (a) The time sequence of the increments from the cases placing a wind x-component innovation of 4 m s−1 at (43.14°N, 141.01°E) at 600 hPa. Increments with the innovation at time step 1 (−180 min; dark blue), time step 91 (−120 min; blue), time step 181 (−60 min; light blue), and time step 271 (0 min; pale blue) for the hybrid 4D-Var; and at time step 1 (−180 min; dark red), time step 91 (−120 min; red), time step 181 (−60 min; light red), and time step 271 (0 min; pale red) for the 4D-Var. (b) The time sequence of the estimated background error standard deviation for the hybrid 4D-Var (blue) and the 4D-Var (red). (c) The estimated background error time correlation between arbitrary time step in the assimilation window and the specified time step. The results are shown for time step 1 (−180 min; dark blue), time step 91 (−120 min; blue), time step 181 (−60 min; light blue), and time step 271 (0 min; pale blue) for the hybrid 4D-Var, and those for time step 1 (−180 min; dark red), time step 91 (−120 min; red), time step 181 (−60 min; light red), and time step 271 (0 min; pale red) for the 4D-Var.

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

Thus, the time correlation of the background error largely depends on the atmospheric condition as well as its spatial correlation. From the discussion in Fowler et al. (2018), it is expected that the hybrid 4D-Var can better reflect small scale temporal variability than the 4D-Var in the former part of the assimilation window, where the climatological B works to give a longer time correlation width in this case. It is, however, noted that, as shown in Fig. 10c, the overall width of the time correlation is large (∼1 h on average for a correlation of 0.4), which limits the temporal resolution of the DA system.

It is inferred that the propagation of the representation error, corresponding to phenomena unresolved by the model, may lead to its time correlation, similar to the propagation of the background error described above.

c. Distribution of the increments

The evolutions of the increments of the U wind component at 600 hPa from both the hybrid 4D-Var (Figs. 11c,d) and the 4D-Var (Figs. 11a,b) show patterns continuous in time from the assimilation window through the 3-h forecast, suggesting increments are in line with the dynamics of the forecast model. However, increments from the hybrid 4D-Var (Figs. 11c,d) have detailed structures from the beginning of the assimilation window (−180 min) reflecting the flow-dependent B, while those from the 4D-Var (Figs. 11a,b) have less structure due to the climatological B, and it remains until the end of the assimilation window (0 min). The same is true for the vertical cross sections (the second from the bottom row). In the power spectrum of the 4D-Var (dark red solid line in Fig. 12a) over the 2560 km × 2560 km subdomain, the components at wavenumber 2 and 3 are dominant at −180 min, and those at larger wavenumbers are small. On the other hand, in the hybrid 4D-Var, the components around a higher wavenumber of 5 are dominant in addition to those at wavenumbers 2 and 3 (dark blue solid line). As time progresses, the components at higher wavenumbers increase for both the 4D-Var (red solid line) and the hybrid 4D-Var (blue solid line). However, the wavenumber of the peak is higher in the hybrid 4D-Var throughout the assimilation window. And even at the end of the assimilation window, the components above wavenumber 4 are larger in the hybrid 4D-Var than in the 4D-Var. These results are consistent with section 5b, and indicate that the hybrid 4D-Var is likely to have a higher potential to extract detailed observational information as a time evolution continuous in time.

Fig. 11.
Fig. 11.

Increments of U wind component of the analysis valid at 0600 UTC 3 Jul 2018. The top three rows display the propagation of increments at 600 hPa every 90 min from the beginning (−180 min) to the end (0 min) of the 3-h assimilation window. The 180-min forecasts are also displayed in the fourth row. The two bottom rows display vertical cross sections of the increments at −180 min along the line denoted in the bottom panel on the left, with pressure contoured every 200 hPa. (a) The 4D-Var with observations assimilated every 60 min. (b) As in (a), but with observations assimilated every 10 min. (c) The hybrid 4D-Var with observations assimilated every 60 min. (d) As in (c), but with observations assimilated every 10 min. The bottom panel of (d) shows the result applying ME on the observation error correlation (see appendix B). Black ellipses in the bottom two panels of (d) indicate the area where differences between these panels are noticeable (see appendix B).

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

Fig. 12.
Fig. 12.

Power spectrum of the analysis increment for a 512 × 512 grid subregion of a 5-km grid spacing on the northeast side of the domain of the outer model. The squared norm of the horizontal two-dimensional discrete Fourier transform is integrated in bins of 1 wavenumber vector length, and then averaged for all the 48 layers of the outer model. (a) Power spectrum at −180 min (dark blue solid line), −120 min (blue solid line), −60 min (light blue solid line), and 0 min (pale blue solid line) for the hybrid 4D-Var with the observation interval of 10 min; and −180 min (dark blue dotted line) for the hybrid 4D-Var with the observation interval of 60 min; −180 min (dark red solid line), −120 min (red solid line), −60 min (light red solid line), and 0 min (pale red solid line) for the 4D-Var with the observation interval of 10 min; and −180 min (dark red dotted line) for the 4D-Var with the observation interval of 60 min. (b) Ratio of the power of the hybrid 4D-Var with the observation interval of 60 min against that of the case with the interval of 10 min. The powers are taken from the corresponding time. Ratio at −180 min (dark blue dotted line), −120 min (blue dotted line), −60 min (light blue dotted line), and 0 min (pale blue dotted line). Gray solid line indicates the ratio 1.

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

The cases with observation interval of 10 min (Figs. 11b,d) give increments over wider ranges than the cases with 60-min interval (Figs. 11a,c) in both of the assimilation schemes. Power spectrum at −180 min (Fig. 12) shows that the experiments with 60-min observation time interval (dark blue and dark red dotted lines) give smaller power than those with 10-min interval (dark blue and dark red solid lines) for most of the wavenumbers. For the hybrid 4D-Var, the ratio of the 60-min interval experiment to the 10-min interval one (Fig. 12b) shows that the difference between the two experiments tends to decrease with time at low wavenumbers below 3 and high wavenumbers above 50. However, the component at wavenumbers 50–100 is still smaller at the end of the assimilation window in the 60-min interval experiment, and at wavenumbers 5–50, where the major amplitudes are located, the ratio remains ∼60% throughout the assimilation window. It can be seen that the assimilation of high-frequency observations reflects a more detailed spatial distribution in the increments.

d. Time evolution of the increments

Figure 13 shows the time evolution of radial wind increments at the elevation angle of 1.1°. Results are also shown from the simple variational DA (section 4) applied on the 2 + 1-dimensional space assimilating innovations at the elevation angle of 1.1° (Fig. 13b).

Fig. 13.
Fig. 13.

Radial wind innovations and increments of the Sapporo radar at the elevation angle of 1.1°. (left) Hourly propagation from the beginning (−180 min) to the end (0 min) of the 3-h assimilation window. (right) The beam range–time cross section at an azimuthal angle of 104.0625° (east-southeast direction) along the black dotted line in the left panels. (a) Innovations. (b) Increments from the simple variational DA (see text for details) with an observation interval of 10 min. (c) Increments from the 4D-Var with an observation interval of 10 min. (d) Increments from the hybrid 4D-Var with an observation interval of 10 min. The size of the cells in the left four columns is 5 km × 5.625° in beam range and azimuthal angle directions. That in the rightmost columns is 5 km × 10 min in beam range and time directions. Black ellipses indicate propagation of the regions discussed in Fig. 15 (see text).

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

The increments from the simple variational assimilation (Fig. 13b) with the small background error correlation width of 15 km reflect the distribution of innovations (Fig. 13a) in detail at observation time. The radial power spectra (Fig. 14a) show that this scheme (black solid and dotted line) gives larger components at wavenumber 2–6 than those of the hybrid 4D-Var from −180 min. The range of wavenumbers giving higher power is consistent with section 4. However, the beam range–time cross sections of the increments along the beam shows that this scheme only partly reflects the flow toward the east (rightmost panels of Fig. 13b), following the distribution pattern of the observations themselves. The temporal power spectrum of this scheme (black solid and dotted line in Fig. 14b) also shows that the components above wavenumber 2 are small. This scheme has no constraint by the model, and the correlation width of the background error is as long as 45 min. Thus, the temporal structure of the increments is limited.

Fig. 14.
Fig. 14.

The power spectrum of radial wind increments of the Sapporo radar at the elevation angle of 1.1° in the cases with the observation interval of 10 min shown in Fig. 13. (a) Radial power spectrum at −180 min (dark blue solid line), −120 min (blue solid line), −60 min (light blue solid line), and 0 min (pale blue solid line) from the hybrid 4D-Var; at −180 min (dark red solid line), −120 min (red solid line), −60 min (light red solid line), and 0 min (pale red solid line) from the 4D-Var; and at −180 min from the simple variational scheme (black solid line). The spectrum is obtained as in Fig. 7. The ratio of the power of the experiments against that of the hybrid 4D-Var. The powers are taken at the corresponding time. Results for the 4D-Var at −180 min (dark red dotted line), −120 min (red dotted line), −60 min (light dotted line), and 0 min (pale red dotted line) and for the simple variational scheme at −180 min (black dotted line) are shown. (b) Temporal power spectrum (solid line) for the hybrid 4D-Var (blue), the 4D-Var (red), and the simple variational scheme (black). The spectrum is obtained by interpolating the 19 time levels at 10-min intervals into 32 points, and averaging the squared norm of the discrete Fourier transform for all the 64 azimuthal angles and 30 beam ranges. The ratio of the power of the experiments against that of the hybrid 4D-Var. Results for the 4D-Var (red dotted line) and the simple variational scheme (black dotted line) are shown. The gray solid line indicates the ratio 1.

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

However, the 4D-Var gives the cross sections of increments (rightmost panels of Fig. 13c) with patterns moving toward east with time, corresponding to the eastward advection of the precipitation system providing radial winds. It is suggested that the time-evolution operator poses a strong dynamical constraint on the optimization. However, the 4D-Var increments do not have detailed structures in the former part of the assimilation window, and a large-scale pattern dominates at −180 min (left panel of Fig. 13c). The radial power spectrum from the 4D-Var (dark red solid line in Fig. 14a) shows that the components at wavenumber 2 and above are considerably smaller compared to those from the hybrid 4D-Var (dark blue solid line) at the −180 min (dark red dotted line). Due to the climatological B, it takes a while for the effect from the time-evolution operator to be dominant enough to reflect flow dependency on the analysis. As time progresses, the high-wavenumber components increase in both the hybrid 4D-Var and the 4D-Var, which can be seen also in the panels from −120 to 0 min in Figs. 13d,c. The 4D-Var shows less components of wavenumbers 2–5 until −60 min, and reaches roughly equivalent to the hybrid 4D-Var at 0 min.

The cross section of the increments from the hybrid 4D-Var (rightmost panels of Fig. 13d) again reflects the eastward advection. Besides, the structure along the eastward flow can be seen from the former part of the assimilation window due to the effect of the flow-dependent B. The increments at −180 min from the hybrid 4D-Var show more detailed structure (left panel of Fig. 13d) than those in the 4D-Var (left panel of Fig. 13c). In the temporal power spectrum (Fig. 14b), the components of wavenumbers 1–4 are larger in the hybrid 4D-Var (blue solid line) than in the 4D-Var (red solid and dotted lines), although the overall temporal resolution is limited. This is consistent with the profile of B discussed in section 5a, where the hybrid 4D-Var showed a smaller temporal correlation width than that of the 4D-Var at −180 min (dark blue and dark red lines in Fig. 10a), expected to allow smaller scale increments in the hybrid 4D-Var. Thus, the hybrid 4D-Var shows its high potential to extract detailed information on the atmospheric flow from observations throughout the assimilation window.

e. Propagation of the increments in forecast

Figure 15 displays the propagated increments of radial winds at 180-min forecast. The hybrid 4D-Var (Fig. 15b) shows larger negative (westward) increments in the east of the Sapporo radar (solid circle), better corresponding to the observations (Fig. 15a) than the 4D-Var (Fig. 15c). This can be traced back to the positive increments located west of the Sapporo radar at 0, −60, and −120 min in Figs. 13c,d (solid circles). The increments from the hybrid 4D-Var are more localized and intensified than those from the 4D-Var. The hybrid 4D-Var shows a potential to effectively extract observational information taking advantage of the flow dependency available throughout the assimilation window.

Fig. 15.
Fig. 15.

Difference of radial winds from the forecast from the first guess for the Sapporo radar at the elevation angle of 1.1°, valid at 0900 UTC 3 Jul 2018. (a) Observation. (b) The 180-min forecast from the hybrid 4D-Var with the observation interval of 10 min. (c) The 180-min forecast from the 4D-Var with the observation interval of 10 min. Black solid lines in (b) and (c) show the distribution of the observations (a). The size of the cells is 5 km × 5.625° in beam range and azimuthal angle directions. Black ellipses indicate regions where the result of the hybrid 4D-Var corresponds more closely to the observations (see text).

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

The Kushiro radar is located at 42.96°N, 144.52°E, ∼280 km east of the Sapporo radar (Fig. 2), lying downstream of the Sapporo site in this case. The hybrid 4D-Var with a 10-min observation interval shows westward wind increments near the Kushiro radar at the elevation angles of 1.8° for the forecast time 0–180 min, which is in better agreement with the observed radial winds than the 60-min interval case (not shown). This corresponds to the wind increments at 850 hPa toward the west decreasing the wind speed in the northern part of the solid yellow circle in Fig. 16a. In the southern part of the circle, there are also wind increments toward the east increasing the wind speed. These increments intensify the wind shear along the front extending over the Kushiro site at 850 hPa (Fig. 16c). The stronger wind shear in the 10-min interval case is likely to contribute to increase rainfall along the front (dashed circle in Fig. 16e). Apart from displacement, the intensified rainfall in the 10-min case is more consistent with the observations (Fig. 16d) than in the 60-min case (Fig. 16f) and in the forecast from the first guess (Fig. 16g). Neither of the 4D-Var cases with the 10- and 60-min observation interval give clear intensification of the wind shear and rainfall (not shown). It is inferred that the hybrid 4D-Var effectively use additional detailed information from the frequent observations to provide a promising result in later forecast in this case.

Fig. 16.
Fig. 16.

(a) Difference of 850-hPa wind (barbs) and wind speed (shaded) between the 60-min forecast from the hybrid 4D-Var with the observation interval of 10 min and that from the first guess, valid at 0700 UTC 3 Jul 2018. (b) As in (a), but for the hybrid 4D-Var with the observation interval of 60 min. (c) The 60-min forecast of 850-hPa wind (barbs) and wind speed (shaded) from the first guess valid at 0700 UTC 3 Jul 2018. (d) Radar–rain gauge analysis of 3-h accumulated rainfall valid at 0900 UTC 3 Jul 2018. (e) The 180-min forecast of 3-h accumulated rainfall valid at 0900 UTC 3 Jul 2018 from the hybrid 4D-Var with the observation interval of 10 min. (f) As in (e), but for the hybrid 4D-Var with the observation interval of 60 min. (g) As in (e), but for the first guess. Short barbs, long barbs, and pennants in (a)–(c) indicate 1, 2, and 10 m s−1, respectively. Locations of the Sapporo and Kushiro radars and the range of assimilated radial winds (150 km from the radar site) are indicated with black pluses and circles, respectively. Yellow solid and black dotted ellipses indicate the area where wind shear and rainfall intensify in the hybrid 4D-Var, respectively (see text).

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

f. Verification of forecasts against radial wind observations

Figure 17 shows the root-mean-square error (RMSE) of the forecast verified against radial winds from the Sapporo and the Kushiro radars. A stationary front brought rainfall to a wide area covering the two radar sites producing radial winds throughout the forecast period. The data from the Kushiro radar are not used in the assimilations, serving as independent verification data during the assimilation window. The QC processes, except for the check against the first guess (item 4 in appendix A, section a), are applied to the radial winds used in the verification.

Fig. 17.
Fig. 17.

RMSE against radial winds through assimilation window and 12-h forecast at an interval of 10 min. The verification includes data from elevation angles 0.1°, 1.1°, 2.6°, and 4.3° for the Sapporo radar and 0.3°, 1.8°, 3.4°, and 5.2° for the Kushiro radar. (a) The Sapporo radar. (b) The Kushiro radar. Dark red: the 4D-Var with an observation interval of 10 min. Light red: the 4D-Var with an observation interval of 60 min. Dark blue: the hybrid 4D-Var with an observation interval of 10 min. Light blue: the hybrid 4D-Var with an observation interval of 60 min. Black: the forecast from the first guess. Gray: number of observations (right axis).

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

For most of the assimilation window, both the 4D-Var (red) and the hybrid 4D-Var (blue) reduced RMSE against the assimilated data from the Sapporo radar (Fig. 17a) compared to the extended forecast from the first guess (black). In the forecast period, the main impact from assimilating radial winds lasts up to ∼6 and ∼8 h in the forecast for the Sapporo and Kushiro site, respectively, for both the hybrid 4D-Var and the 4D-Var.

The hybrid 4D-Var, with the observation interval of 10 min (dark blue), tends to give the smallest RMSE, consistently smaller than the 60-min case (light blue) for most of the forecast period. However, the advantage of the 10-min interval case is not clear in the 4D-Var. Although we need to investigate more cases to reach a conclusion, it is suggested that the flow-dependent B of the hybrid 4D-Var may help to get more advantage from the use of high-frequency observations.

To investigate the effect of the temporal and spatial correlations of the observation error, sensitivity experiments are conducted. In the 4D-Var (Figs. 18a,b), neglecting the temporal and spatial correlations (dark red solid lines) degrades the forecast. For the Kushiro radar (Fig. 18b), the RMSE is often even larger than that of the first guess (black solid line). The case without temporal correlation (light red solid line) and that with a doubled temporal correlation width (light red dotted line) improve the results, but also underperform the case with the full correlation (red solid line). These results suggest that it is necessary for the 4D-Var to properly consider the temporal and spatial correlation of the observation error in the assimilation of high-frequency and high-density observation data.

Fig. 18.
Fig. 18.

As in Fig. 17, but for sensitivity experiments for the observation error correlation. (a) The 4D-Var for the Sapporo radar. (b) The 4D-Var for the Kushiro radar. (c) The hybrid 4D-Var for the Sapporo radar. (d) The hybrid 4D-Var for the Kushiro radar. The results from the hybrid 4D-Var are shown in the following line types. Blue solid line: the full correlation, 10-min observation interval. Dark blue solid line: no correlation, 10-min interval. Light blue solid line: no temporal correlation, 10-min interval. Light blue dotted line: doubled temporal correlation width, 10-min interval. Dark blue dotted line: no correlation, 60-min interval. Blue dotted line: spatial and temporal correlation width of 0.7 times, 10-min interval. The corresponding line type in red shows the results from the 4D-Var. Black solid line: the first guess.

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

When the temporal and spatial correlations are neglected, the 60-min observation interval case (dark red dotted lines) gives smaller RMSE than the 10-min case (dark red solid line), which is reasonable because the correlation becomes smaller when the observation time interval is increased. RMSE is even smaller in the 10-min case with the full correlation (solid red lines), suggesting that an effective use of frequent observations is achieved by appropriate handling of the observation error correlation.

On the other hand, for the hybrid 4D-Var (Figs. 18c,d), neglecting the observation error correlation does not necessarily lead to degradation of the forecast (dark blue and light blue solid lines), suggesting that the flow-dependent B has a large contribution to the prediction accuracy.

When the temporal and spatial correlations are neglected, the RMSE tends to be smaller in the 60-min observation interval case (dark blue dotted lines) than in the 10-min case (dark blue solid lines), which is reasonable as discussed in the case of the 4D-Var. However, the RMSE (dark blue dotted lines) tends to be smaller than that of the 10-min case with the full correlation (blue solid lines). The RMSE of the latter case (blue solid lines) is large especially for the Kushiro radar in the 2–5-h forecast, and the doubled temporal correlation gives even larger RMSE (light blue dotted line). These results suggest that the modeling of the observation error may not be optimal degrading the forecast.

As shown in section 3d, the diagnosed correlation width of the observation error is smaller when the flow-dependent B is used in the DA cycle (Fig. 5). Based on this, we tried the 10-min case with temporal and spatial correlation widths of 0.7 times (blue dotted lines), reducing the overestimation at short separations (gray dotted lines in Fig. 5). This works to reduce RMSE in the 2–5-h forecast for the Kushiro radar [however, the precipitation discussed in Fig. 16e is weakened (not shown)]. It is suggested that there is room for refinement in the modeling of the observation errors. This study is based on a single case study using data from a single radar site with a short statistical period. More case studies are needed to draw conclusions about performance. Further investigation is a future work.

6. Discussion and conclusions

Aimed at an effective use of high-density, high-frequency observations, the present study explored the use of radial winds to enhance NWP from the R and B as major components of DA.

The statistical diagnosis shows an observation error correlation width of ∼15 km, 15 km, and 45 min in beam range, azimuthal, and time directions, respectively. The diagnosed spatial width is considerably larger than that expected from the instrument error, suggesting a large contribution from the errors in the observation operators and the representation errors (Janjić et al. 2018) associated with phenomena that cannot be represented in the DA system. The results are mostly consistent with diagnoses in various systems using the same method (Wattrelot et al. 2012; Waller et al. 2016c, 2019; Zeng et al. 2021). The width of the temporal correlation increases with the forecast time, showing that the time evolution of the model is related to the correlation. The propagation of the representation error is considered to contribute to the correlation. Another diagnosis using the flow-dependent B in the DA cycle gives smaller variance and correlation width, suggesting that the resolution of increments increases and the representation error at larger scales reduces.

Our experimentation with a simplified variational scheme shows that incorporating observation error correlation allowed us to consistently bring the detailed structure of innovations into the analysis. Consistent with Fowler et al. (2018), the observation error correlation results in a stronger constraint on the high-wavenumber components. However, the results from the simple variational scheme are snapshots that disregard time evolution.

The strong constraint from the model dynamics of the 4D-Var resulted in a drastic change of the analysis increments, dealing with the distribution of observations in time and space as a series of time evolution. Simultaneously, the 4D-Var showed limitations caused by the climatological B, resulting in poor resolution especially during the former part of the assimilation window.

The hybrid 4D-Var, implemented with the flow-dependent B, showed more detailed, time continuous structures, suggesting its high potential to extract detailed information from observations using the full assimilation window. The single observation experiment suggests that larger background error at higher wavenumbers in the hybrid 4D-Var leads to the increase in increments at higher wavenumbers. The forecast from the hybrid 4D-Var also gives promising results.

Sensitivity experiments showed that, in the 4D-Var, neglecting the error correlation degraded the forecast in terms of RMSE against radial winds, showing that the appropriate consideration of the observation error correlation leads to the effective use of high-frequency observations. On the other hand, for the hybrid 4D-Var, neglecting the observation error correlation did not necessarily degrade the forecast, suggesting a significant contribution of the flow-dependent B to the forecasting characteristics. The 10-min observation interval case with the observation error correlation outperformed the 60-min case without the correlation for the 4D-Var, showing the effect of the correlated observation error. However, it underperformed in the hybrid 4D-Var. Using a reduced correlation width, which is expected to be more consistent with the diagnosis with the hybrid 4D-Var, improved the forecast of the 10-min case to reduce the difference. The optimal modeling of the observation error correlation needs to be further explored with statistics for more cases.

We recognize that some aspects of this research contain needed simplifications. The observation error correlation was diagnosed from statistics for only one week of data from a single radar site. Ideally, the statistics need to include various cases under different atmospheric conditions and from different radar sites to get the more general properties of the correlation. The impact study was also conducted using a single case. An extended investigation is needed for a quantitative assessment of impacts on forecast accuracy in the future. It is also required to optimize the configuration of the ensemble used in the hybrid 4D-Var to generate a high-quality flow-dependent B to extract sufficient information from the dense observations.

Therefore, further research is required to make a robust evaluation. However, our results suggest that the flow-dependent background error is essential for the extensive use of high-density, high-frequency observations.

Acknowledgments.

The authors thank Drs. Ken Sawada, Daisuke Hotta, and Yasutaka Ikuta in the Meteorological Research Institute for helpful discussions. The authors are grateful to the three anonymous reviewers for their insightful comments that helped us improve the paper. This work was supported by JSPS KAKENHI Grants JP19K23467 and JP21K03667, JST AIP Grant JPMJCR19U2, and “Program for Promoting Researches on the Supercomputer Fugaku” (Large Ensemble Atmospheric and Environmental Prediction for Disaster Prevention and Mitigation, ID:hp200128/hp210166) of MEXT (JPMXP1020200305). This work is based on the former operational NWP system developed by Numerical Prediction Division, Japan Meteorological Agency. The authors thank Enago (www.enago.jp) for the English language review.

Data availability statement.

The output data from this study have been archived and are available upon request to the corresponding author. The observational data and the data assimilation system are made available under a contract with Japan Meteorological Agency, because these are basically collected and developed for the operational purpose.

APPENDIX A

The Quality Control and Observation Operator of Radial Winds in MA

This appendix describes the handling of radial wind data in MA (Ishikawa and Koizumi 2006; Seko et al. 2004; JMA 2019). JMA operates the weather radar network consisting of 20 C-band weather Doppler radars and the aviation weather observation system including 9 C-band airport weather Doppler radars. Radial winds from these radars, along with reflectivities and radar/rain gauge-analyzed precipitations are used in MA as important remote sensing data. The radial winds originally are measured at a resolution of 250 m (weather Doppler radar) or 150 m (aviation weather Doppler radar) in the radial direction and 0.703° in the azimuthal direction. They are averaged over cells with a size of 5 km × 5.625° for use in MA. Hourly data on the hour are used out of data measured at a time interval of 5 min (weather Doppler radar; lower elevation angles used in MA have two scans during a 10-min scan sequence) or ∼6 min (aviation weather Doppler radar).

a. Quality control

QC processes are applied on the cell-averaged data, rejecting the data if either of the following check items apply.

  1. The number of the cell samples is less than 10.

  2. The standard deviation of the cell samples is larger than or equal to 10 m s−1.

  3. The difference between the maximum and minimum cell-sample is larger than or equal to 10 m s−1.

  4. The cell-average departs from the first guess over 10 m s−1.

  5. The distance from the radar site is less than 15 km.

  6. The elevation angle is larger than or equal to 5.9°.

  7. The cell-average is less than 5 m s−1.

  8. The cell-average departs from the average of surrounding data over 10 m s−1.

b. Observation operator

The observation operator of the radial wind consists of the following processes.

  • Horizontal winds in each of the four vertical columns surrounding the observation point are vertically averaged within the width of the radar beam. The beam intensity is assumed to have a Gaussian form in vertical direction with a width of 0.3° peaked at the beam center.

  • The vertically averaged winds from the four columns are horizontally interpolated to the observation point.

  • The wind component tangential to the radar beam is derived as the radial wind.

The height of the observation point is assigned according to the elevation angle and the radial distance from the radar site considering the altitude of the radar instrument. The estimation uses the effective Earth’s radius model to take into account the earth curvature and the refractive index of the atmosphere, applying 4/3 Earth’s radius (Doviak and Zrnić 1993).

APPENDIX B

Conditioning of the Observation Error Covariance Matrices

This appendix surveys the effect of the truncation of eigenmodes applied on the observation error correlation in section 3c.

The observation error correlation is modeled using a Gaussian function, whose eigenvalues rapidly decrease in size when listed in decreasing order. For radial winds from the Sapporo radar at the elevation angle of 1.1° from 0310 to 0600 UTC 3 July 2018, the sum of the eigenvalues reaches 99% of the trace with 515 of the 10 621 eigenmodes (the condition number is 786.5). Taking modes up to 99% of the trace, the correlation itself is approximated with high accuracy (not shown). On the other hand, the contribution of each eigenmode to the cost function (Fig. B1a black) is proportional to the inverse of the eigenvalue, showing an exponential increase as the eigenvalue becomes smaller (small negative eigenvalues occur at the 9532nd eigenvalue due to the limitation of calculation accuracy). Since larger (smaller) eigenvalues generally correspond to eigenmodes with lower (higher) wavenumber, this indicates a strong constraint on eigenmodes with higher wavenumbers. In the mode truncation, this constraint on higher wavenumber modes is excluded, and the cost function is greatly reduced (blue). While the mode truncation decreases the rank of R, there are two methods to improve the condition number retaining the rank of R, which are discussed in Tabeart et al. (2020):

  • Minimum eigenvalue (ME) method: Replace small eigenvalues with a constant.

  • Ridge regression (RR) method: Add a constant to the diagonal elements.

Fig. B1.
Fig. B1.

The profiles of cost function and innovation of radial wind decomposed by the eigenmodes of the observation error correlation. Results are shown for different reconditioning methods of R. Radial winds are from the Sapporo radar at the elevation angle of 1.1° from 0310 to 0600 UTC 3 Jul 2018. (a) Contribution from the eigenmodes to the radial wind observation term of the initial cost function. (b) Change in the norm of the eigenmode components of the normalized radial wind residuals through the assimilation. Black: without reconditioning. Blue: the truncation of the eigenmode. Dark blue: the minimum eigenvalue method. Light blue: the ridge regression method. Dotted line in (b) indicates the position of the truncation (eigenmode number 515).

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

The ME (dark blue) and RR (light blue) for the condition number 786.5 show almost similar behavior, although they differ slightly around the truncation. While all eigenmodes are retained in both methods, these methods are similar to the mode truncation for the Gaussian correlation in that they work to greatly weaken the constraint on high-wavenumber modes.

The change in each eigenmode component of residuals before and after the assimilation, |vkTd^a|2|vkTd^b|2, is shown in Fig. B1b, where d^a and d^b are the analysis and the background residuals normalized by the observation error standard deviation, respectively. The vk is the eigenvector of the kth eigenmode of the observation error correlation, and the superscript T denotes transpose. The variational optimization overall reduces the residuals. On the other hand, in the DA system in this study, the amplitude of the response to assimilation is mostly dominated by components with larger eigenvalues, and the range is largely covered by the eigenmodes available after the truncation. The high-wavenumber eigenmodes have a large contribution in the cost function, but their contribution to the physical amplitude of increments is small. In optimization, the mode truncation method puts more weight on the eigenmodes up to the truncation (the cost function from these modes is optimized from an initial 563.65 to a final 348.39 for the mode truncation and a final 369.18 for the ME). Thus, although the mode truncation and the ME give overall consistent distribution of the increments (bottom two panels in Fig. 11d), there are some differences at large scales (solid circles). On the other hand, there is no clear difference in performance in the RMSE against radial winds (Figs. B2a,b). Thus, in the DA system and the modeling of R of this study, it is suggested that the mode truncation does not have a significant influence on the performance compared to ME and RR.

Fig. B2.
Fig. B2.

As in Fig. 17, but for different reconditioning methods of R. (a) Sapporo radar, (b) Kushiro radar. Blue: the truncation of the eigenmode. Dark blue: the minimum eigenvalue method. Light blue: the ridge regression method. Black: the first guess.

Citation: Monthly Weather Review 150, 3; 10.1175/MWR-D-21-0121.1

Future works include the refinement of the modeling of R based on more reliable statistics with more cases.

REFERENCES

  • Bédard, J., and M. Buehner, 2020: A practical assimilation approach to extract smaller-scale information from observations with spatially correlated errors: An idealized study. Quart. J. Roy. Meteor. Soc., 146, 468482, https://doi.org/10.1002/qj.3687.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bennitt, G. V., H. R. Johnson, P. P. Weston, J. Jones, and E. Pottiaux, 2017: An assessment of ground-based GNSS zenith total delay observation errors and their correlations using the Met Office UKV model. Quart. J. Roy. Meteor. Soc., 143, 24362447, https://doi.org/10.1002/qj.3097.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bormann, N., and P. Bauer, 2010: Estimates of spatial and interchannel observation-error characteristics for current sounder radiances for numerical weather prediction. I: Methods and application to ATOVS data. Quart. J. Roy. Meteor. Soc., 136, 10361050, https://doi.org/10.1002/qj.616.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bormann, N., A. Collard, and P. Bauer, 2010: Estimates of spatial and interchannel observation-error characteristics for current sounder radiances for numerical weather prediction. II: Application to AIRS and IASI data. Quart. J. Roy. Meteor. Soc., 136, 10511063, https://doi.org/10.1002/qj.615.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bormann, N., M. Bonavita, R. Dragani, R. Eresmaa, M. Matricardi, and A. McNally, 2016: Enhancing the impact of IASI observations through an updated observation-error covariance matrix. Quart. J. Roy. Meteor. Soc., 142, 17671780, https://doi.org/10.1002/qj.2774.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bryson, A. E., and L. J. Henrikson, 1968: Estimation using sampled data containing sequentially correlated noise. J. Spacecr. Rockets, 5, 662665, https://doi.org/10.2514/3.29327.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buehner, M., 2005: Ensemble-derived stationary and flow-dependent background-error covariances: Evaluation in a quasi-operational NWP setting. Quart. J. Roy. Meteor. Soc., 131, 10131043, https://doi.org/10.1256/qj.04.15.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Campbell, W. F., E. A. Satterfield, B. Ruston, and N. L. Baker, 2017: Accounting for correlated observation error in a dual-formulation 4D variational data assimilation system. Mon. Wea. Rev., 145, 10191032, https://doi.org/10.1175/MWR-D-16-0240.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Courtier, P., J.-N. Thépaut, and A. Hollingsworth, 1994: A strategy for operational implementation of 4D-Var, using an incremental approach. Quart. J. Roy. Meteor. Soc., 120, 13671387, https://doi.org/10.1002/qj.49712051912.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Desroziers, G., L. Berre, B. Chapnik, and P. Poli, 2005: Diagnosis of observation, background and analysis-error statistics in observation space. Quart. J. Roy. Meteor. Soc., 131, 33853396, https://doi.org/10.1256/qj.05.108.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Doviak, R. J., and D. S. Zrnić, 1993: Doppler Radar and Weather Observations. 2nd ed. Academic Press, 562 pp.

  • Evensen, G., and K. S. Eikrem, 2018: Conditioning reservoir models on rate data using ensemble smoothers. Comput. Geosci., 22, 12511270, https://doi.org/10.1007/s10596-018-9750-8.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fowler, A. M., 2019: Data compression in the presence of observational error correlations. Tellus, 71A, 1634937, https://doi.org/10.1080/16000870.2019.1634937.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fowler, A. M., S. L. Dance, and J. A. Waller, 2018: On the interaction of observation and prior error correlations in data assimilation. Quart. J. Roy. Meteor. Soc., 144, 4862, https://doi.org/10.1002/qj.3183.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Gustafsson, N., and Coauthors, 2018: Survey of data assimilation methods for convective-scale numerical weather prediction at operational centres. Quart. J. Roy. Meteor. Soc., 144, 12181256, https://doi.org/10.1002/qj.3179.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Hólm, E. V., S. T. K. Lang, M. Fisher, T. Kral, and M. Bonavita, 2018: Distributed observations in meteorological ensemble data assimilation and forecasting. Proc. 21st Int. Conf. on Information Fusion (FUSION), Cambridge, United Kingdom, Institute of Electrical and Electronics Engineers, 9299, https://doi.org/10.23919/ICIF.2018.8455209.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Honda, Y., M. Nishijima, K. Koizumi, Y. Ohta, K. Tamiya, T. Kawabata, and T. Tsuyuki, 2005: A pre-operational variational data assimilation system for a non-hydrostatic model at the Japan Meteorological Agency: Formulation and preliminary results. Quart. J. Roy. Meteor. Soc., 131, 34653475, https://doi.org/10.1256/qj.05.132.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Isaksen, L., M. Bonavita, R. Buizza, M. Fisher, J. Haseler, M. Leutbecher, and L. Raynaud, 2010: Ensemble of data assimilations at ECMWF. ECMWF Tech. Memo. 636, ECMWF, https://doi.org/10.21957/obke4k60.

    • Search Google Scholar
    • Export Citation
  • Ishikawa, Y. and K. Koizumi, 2006: Doppler radar wind data assimilation with the JMA Meso 4D-VAR. Research activities in atmospheric and oceanic modelling. CAS/JSC WGNE, 36, 01.11–01.12, https://www.wcrp-climate.org/WGNE/BlueBook/2006/individual-articles/01_Ishikawa_Yoshihiro_Doppler_Meso_4D-Var.pdf.

    • Search Google Scholar
    • Export Citation
  • Janjić, T., and Coauthors, 2018: On the representation error in data assimilation. Quart. J. Roy. Meteor. Soc., 144, 12571278, https://doi.org/10.1002/qj.3130.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Järvinen, H., E. Andersson, and F. Bouttier, 1999: Variational assimilation of time sequences of surface observations with serially correlated errors. Tellus, 51A, 469488, https://doi.org/10.3402/tellusa.v51i4.13963.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • JMA, 2019: Outline of the operational numerical weather prediction at the Japan Meteorological Agency. Appendix to WMO Technical Progress Report on the global data-processing and forecasting system and numerical weather prediction, Japan Meteorological Agency, 229 pp., https://www.jma.go.jp/jma/jma-eng/jma-center/nwp/outline2019-nwp/index.htm.

    • Search Google Scholar
    • Export Citation
  • Kawabata, T., H. Seko, K. Saito, T. Kuroda, K. Tamiya, T. Tsuyuki, Y. Honda, and Y. Wakazuki, 2007: An assimilation and forecasting experiment of the Nerima heavy rainfall with a cloud-resolving nonhydrostatic 4-dimensional variational data assimilation system. J. Meteor. Soc. Japan, 85, 255276, https://doi.org/10.2151/jmsj.85.255.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Kawabata, T., T. Kuroda, H. Seko, and K. Saito, 2011: A cloud-resolving 4DVAR assimilation experiment for a local heavy rainfall event in the Tokyo metropolitan area. Mon. Wea. Rev., 139, 19111931, https://doi.org/10.1175/2011MWR3428.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Liu, Z.-Q., and F. Rabier, 2002: The interaction between model resolution, observation resolution and observation density in data assimilation: A one-dimensional study. Quart. J. Roy. Meteor. Soc., 128, 13671386, https://doi.org/10.1256/003590002320373337.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Miyoshi, T., E. Kalnay, and H. Li, 2013: Estimating and including observation-error correlations in data assimilation. Inverse Probl. Sci. Eng., 21, 387398, https://doi.org/10.1080/17415977.2012.712527.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Pinnington, E. M., E. Casella, S. L. Dance, A. S. Lawless, J. I. L. Morison, N. K. Nichols, M. Wilkinson, and T. L. Quaife, 2016: Investigating the role of prior and observation error correlations in improving a model forecast of forest carbon balance using four-dimensional variational data assimilation. Agric. For. Meteor., 228–229, 299314, https://doi.org/10.1016/j.agrformet.2016.07.006.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Rainwater, S., C. H. Bishop, and W. F. Campbell, 2015: The benefits of correlated observation errors for small scales. Quart. J. Roy. Meteor. Soc., 141, 34393445, https://doi.org/10.1002/qj.2582.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Saito, K., and Coauthors, 2006: The operational JMA nonhydrostatic mesoscale model. Mon. Wea. Rev., 134, 12661298, https://doi.org/10.1175/MWR3120.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Seko, H., T. Kawabata, T. Tsuyuki, H. Nakamura, K. Koizumi, and T. Iwabuchi, 2004: Impacts of GPS-derived water vapor and radial wind measured by Doppler radar on numerical prediction of precipitation. J. Meteor. Soc. Japan, 82, 473489, https://doi.org/10.2151/jmsj.2004.473.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Simonin, D., J. A. Waller, S. P. Ballard, S. L. Dance, and N. K. Nichols, 2019: A pragmatic strategy for implementing spatially correlated observation errors in an operational system: An application to Doppler radial winds. Quart. J. Roy. Meteor. Soc., 145, 27722790, https://doi.org/10.1002/qj.3592.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stewart, L. M., S. L. Dance, and N. K. Nichols, 2013: Data assimilation with correlated observation errors: Experiments with a 1-D shallow water model. Tellus, 65A, 19546, https://doi.org/10.3402/tellusa.v65i0.19546.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Stewart, L. M., S. L. Dance, N. K. Nichols, J. R. Eyre, and J. Cameron, 2014: Estimating interchannel observation-error correlations for IASI radiance data in the Met Office system. Quart. J. Roy. Meteor. Soc., 140, 12361244, https://doi.org/10.1002/qj.2211.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Tabeart, J. M., S. L. Dance, A. S. Lawless, N. K. Nichols, and J. A. Waller, 2020: Improving the condition number of estimated covariance matrices. Tellus, 72A, 119, https://doi.org/10.1080/16000870.2019.1696646.

    • Search Google Scholar
    • Export Citation
  • Terasaki, K., and T. Miyoshi, 2014: Data assimilation with error-correlated and non-orthogonal observations: Experiments with the Lorenz-96 model. SOLA, 10, 210213, https://doi.org/10.2151/sola.2014-044.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Waller, J. A., S. P. Ballard, S. L. Dance, G. Kelly, N. K. Nichols, and D. Simonin, 2016a: Diagnosing horizontal and inter-channel observation error correlations for SEVIRI observations using observation-minus-background and observation-minus-analysis statistics. Remote Sens., 8, 581, https://doi.org/10.3390/rs8070581.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Waller, J. A., S. L. Dance, and N. K. Nichols, 2016b: Theoretical insight into diagnosing observation error correlations using observation-minus-background and observation-minus-analysis statistics. Quart. J. Roy. Meteor. Soc., 142, 418431, https://doi.org/10.1002/qj.2661.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Waller, J. A., D. Simonin, S. L. Dance, N. K. Nichols, and S. P. Ballard, 2016c: Diagnosing observation error correlations for Doppler radar radial winds in the Met Office UKV model using observation-minus-background and observation-minus-analysis statistics. Mon. Wea. Rev., 144, 35333551, https://doi.org/10.1175/MWR-D-15-0340.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Waller, J. A., E. Bauernschubert, S. L. Dance, N. K. Nichols, R. Potthast, and D. Simonin, 2019: Observation error statistics for Doppler radar radial wind super observations assimilated into the DWD COSMO-KENDA system. Mon. Wea. Rev., 147, 33513364, https://doi.org/10.1175/MWR-D-19-0104.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Wattrelot, E., T. Montmerle, and C. G. Guerrero, 2012: Evolution of the assimilation of radar data in the AROME model at convective scale. Proc. Seventh European Conf. on Radar in Meteorology and Hydrology, ERAD-12, Toulouse, France, Météo-France, http://www.meteo.fr/cic/meetings/2012/ERAD/extended_abs/NWP_401_ext_abs.pdf.

    • Search Google Scholar
    • Export Citation
  • Weston, P. P., W. Bell, and J. R. Eyre, 2014: Accounting for correlated error in the assimilation of high-resolution sounder data. Quart. J. Roy. Meteor. Soc., 140, 24202429, https://doi.org/10.1002/qj.2306.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xu, Q., K. Nai, and L. Wei, 2007a: An innovation method for estimating radar radial-velocity observation error and background wind error covariances. Quart. J. Roy. Meteor. Soc., 133, 407415, https://doi.org/10.1002/qj.21.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Xu, Q., K. Nai, L. Wei, H. Lu, P. Zhang, S. Liu, and D. Parrish, 2007b: Estimating radar wind observation error and NCEP WRF background wind error covariances from radar radial-velocity innovations. 18th Conf. on Numerical Weather Prediction, Park City, UT, Amer. Meteor. Soc., 1B.3, https://ams.confex.com/ams/pdfpapers/123419.pdf.

    • Search Google Scholar
    • Export Citation
  • Zeng, Y., T. Janjić, Y. Feng, U. Blahak, A. de Lozar, E. Bauernschubert, K. Stephan, and J. Min, 2021: Interpreting estimated observation error statistics of weather radar measurements using the ICON-LAM-KENDA system. Atmos. Meas. Tech., 14, 57355756, https://doi.org/10.5194/amt-14-5735-2021.

    • Crossref
    • Search Google Scholar
    • Export Citation
Save
  • Bédard, J., and M. Buehner, 2020: A practical assimilation approach to extract smaller-scale information from observations with spatially correlated errors: An idealized study. Quart. J. Roy. Meteor. Soc., 146, 468482, https://doi.org/10.1002/qj.3687.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bennitt, G. V., H. R. Johnson, P. P. Weston, J. Jones, and E. Pottiaux, 2017: An assessment of ground-based GNSS zenith total delay observation errors and their correlations using the Met Office UKV model. Quart. J. Roy. Meteor. Soc., 143, 24362447, https://doi.org/10.1002/qj.3097.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bormann, N., and P. Bauer, 2010: Estimates of spatial and interchannel observation-error characteristics for current sounder radiances for numerical weather prediction. I: Methods and application to ATOVS data. Quart. J. Roy. Meteor. Soc., 136, 10361050, https://doi.org/10.1002/qj.616.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bormann, N., A. Collard, and P. Bauer, 2010: Estimates of spatial and interchannel observation-error characteristics for current sounder radiances for numerical weather prediction. II: Application to AIRS and IASI data. Quart. J. Roy. Meteor. Soc., 136, 10511063, https://doi.org/10.1002/qj.615.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bormann, N., M. Bonavita, R. Dragani, R. Eresmaa, M. Matricardi, and A. McNally, 2016: Enhancing the impact of IASI observations through an updated observation-error covariance matrix. Quart. J. Roy. Meteor. Soc., 142, 17671780, https://doi.org/10.1002/qj.2774.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Bryson, A. E., and L. J. Henrikson, 1968: Estimation using sampled data containing sequentially correlated noise. J. Spacecr. Rockets, 5, 662665, https://doi.org/10.2514/3.29327.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Buehner, M., 2005: Ensemble-derived stationary and flow-dependent background-error covariances: Evaluation in a quasi-operational NWP setting. Quart. J. Roy. Meteor. Soc., 131, 10131043, https://doi.org/10.1256/qj.04.15.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Campbell, W. F., E. A. Satterfield, B. Ruston, and N. L. Baker, 2017: Accounting for correlated observation error in a dual-formulation 4D variational data assimilation system. Mon. Wea. Rev., 145, 10191032, https://doi.org/10.1175/MWR-D-16-0240.1.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Courtier, P., J.-N. Thépaut, and A. Hollingsworth, 1994: A strategy for operational implementation of 4D-Var, using an incremental approach. Quart. J. Roy. Meteor. Soc., 120, 13671387, https://doi.org/10.1002/qj.49712051912.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Desroziers, G., L. Berre, B. Chapnik, and P. Poli, 2005: Diagnosis of observation, background and analysis-error statistics in observation space. Quart. J. Roy. Meteor. Soc., 131, 33853396, https://doi.org/10.1256/qj.05.108.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Doviak, R. J., and D. S. Zrnić, 1993: Doppler Radar and Weather Observations. 2nd ed. Academic Press, 562 pp.

  • Evensen, G., and K. S. Eikrem, 2018: Conditioning reservoir models on rate data using ensemble smoothers. Comput. Geosci., 22, 12511270, https://doi.org/10.1007/s10596-018-9750-8.

    • Crossref
    • Search Google Scholar
    • Export Citation
  • Fowler, A. M., 2019: Data compression in the presence of observational er