1. Introduction
Geostationary meteorological satellites have been used to improve the knowledge and description of atmospheric flow, especially over the large ocean regions that are void of traditional land-based observations. Atmospheric motion vectors (AMVs) derived from tracking clouds and water vapor in sequences of imagery have established themselves as an integral part of the global observation system essential to medium-range weather forecasting (e.g., Kelly 1993) and also for the prediction of severe weather (e.g., Velden 1996). The operational schemes that derive AMVs are now fully automated and operate on observations in the visible (VIS) and infrared (IR) window as well as water vapor (WV) absorption bands. Typically, thousands of cloud and moisture features are selected in the first image of the sequence and then followed in the subsequent images with automated matching techniques such as cross correlation. The height assignment is generally based on the temperature observed in the infrared spectrum and is corrected for semitransparency and atmospheric absorption. Detailed descriptions of the current operational schemes can be found in Nieman et al. (1997), Velden et al. (1997), Bhatia et al. (1996), Tokuno (1996), and Schmetz et al. (1993). The current automated procedures can provide frequent sampling of vector coverage over a broad spatial domain at high density. It is also imperative to employ some internal quality control at the data processing stage to remove poor vectors that do not represent the instantaneous atmospheric flow. The large amount of vectors that are currently derived is too demanding for manual editing. Therefore, emphasis has been placed on the research and development of robust automated quality control (AQC) procedures that are capable of removing suspect vectors related to tracking, height assignment, and tracer representation errors. Furthermore, these schemes are being designed to provide a quality estimate for each individual displacement vector, as well as to provide information on how representative these vectors are to instantaneous motion at a single tropospheric level. These quality estimates can be employed by the user community to select the part of the vector field that best suits their application, as well as in data assimilation schemes for optimizing the data selection procedures.
The current operational AMV extraction centers all invoke AQC schemes. Traditional schemes (e.g., Le Marshall et al. 1994; Bhatia et al. 1996; Tokuno 1996;Nieman et al. 1997) are usually based on vector acceleration checks and simple thresholding techniques that compare the derived vectors to their surrounding vectors or to collocated forecast fields. All vectors that show an acceleration, directional deviation, or discrepancy to other observations larger than a predefined value are rejected. This thresholding approach is fairly successful as a gross error check, but it does not provide further information on the quality of the vectors passed on to the user. Therefore, advanced methodologies have been developed at the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT) and the University of Wisconsin–Cooperative Institute for Meteorological Satellite Studies (UW–CIMSS). The EUMETSAT scheme utilizes tests similar to those applied in the thresholding techniques; they are, however, implemented as normalized continuous functions, providing test results that can be combined into a final quality indicator (Holmlund 1998). At UW–CIMSS, the AQC is controlled by a series of quality checks and a three-dimensional objective analysis scheme referred to as the auto-editor (AE; Hayden and Purser 1995; Velden et al. 1998), which results in the assignment of vector quality flags.
This paper will give a brief overview of the AQC schemes developed at EUMETSAT and UW–CIMSS that are currently employed at their respective national AMV extraction centers. Results from intercomparison studies will quantitatively show the strengths and weaknesses of the two respective approaches. A new combined quality control approach has been developed that takes advantage of the strengths of each method. This scheme is presented together with validation results based on collocated rawinsonde data and European Centre for Medium-Range Weather Forecasts (ECMWF) model forecast impact studies. Sections 2 and 3 describe the two AQC schemes. Section 4 presents the validation results based on previous work leading to the research aimed at a combined quality control approach. Section 5 presents validation results for the two quality control schemes based on a common dataset during the North Pacific Experiment (NORPEX) field campaign. Section 6 shows the results of the ECMWF forecast impact study performed with the new combined AQC method. Finally, section 7 presents the conclusions.
2. The EUMETSAT AQC scheme
The EUMETSAT AQC scheme is described in Holmlund (1998). The algorithm configuration is based on five different vector tests described in Table 1.
The tests are normalized with a simple tanh function that returns values between 0 and 1, where 0 indicates poor quality and 1 high quality. The tuning of the individual tests has been achieved by deriving statistics against collocated rawinsondes as well as by qualitative evaluation performed by experienced shift meteorologists. Based on this tuning, the spatial test has been given a higher weight than the other tests. The current tests have been tuned to perform in a binary-like mode, that is, most tests return values preferably close to 0 or close to 1. Figure 1 shows the frequency histogram of the different tests derived 0900 GMT 7 February 1998. It can be seen that for the case in question, for which 19 662 vectors were produced, between 2000 and 6000 vectors received a quality close to 0 for all tests and 2000 to 7000 received a quality close to 1. Only the forecast test behaved differently by returning a significantly smaller fraction (around 500) of quality values above 0.95.
In addition to the tests described above, the QI scheme also involves an interchannel consistency check. This check compares low-level IR and VIS winds to corresponding clear-sky water vapor winds. It has been infrequently observed that the cloud classification scheme applied at EUMETSAT (e.g., Tomassini 1981) fails to identify very thin cirrus clouds. The target matching methods do, however, sometimes pick up these clouds, hence following the upper-tropospheric flow. The height assignment is, however, only based on the uncorrected infrared temperature of the target and gives an erroneous result. The fact that the radiance observed in the 6.3-μm water vapor channel mainly originates above 600 hPa, the derived clear-sky WV winds only describe mid- and high-level motion. Therefore the low-level IR and VIS winds should describe a completely different motion than the clear-sky WV winds, and consequently if the two sets of winds do describe similar motion one of the two sets (low-level IR/VIS or clear-sky WV winds) has to be wrong. As the clear-sky WV winds cannot detect low-level motion, the conclusion is that the IR/VIS winds are related to extremely thin cirrus that have remained unidentified in the image analysis and are therefore erroneously assigned to a low level. However, under some circumstances the low-level flow is similar to the mid- and upper-tropospheric flow, and in these cases this test can remove correct winds. In order to minimize this problem the check is applied only to low-level winds faster than a certain threshold. Table 2 shows the root-mean-square (rms) vector difference normalized by the mean wind speed (nrms) for low-level winds (below 700 hPa) in different speed classes derived against collocated rawinsondes during September–October 1998. During the period in question only winds faster than 5 m s−1 were disseminated.
Table 2 shows an increase in nrms for winds faster than 15 m s−1 that indicates that the quality of the vectors is decreasing (see section 4a). These winds compose 14% of the total yield during the period in question. Typically a low-level wind with a speed of 15 m s−1 is removed if the vector difference to the collocated clear-sky WV winds is within 2 m s−1. This test generally removes 1%–2% of the derived low-level winds and further analysis has shown that roughly 0.1% of the removed vectors were indeed good. The threshold 15 m s−1 has therefore been selected as the operational threshold as a good compromise between loss of good vectors and removal of low-level wind vectors with gross errors.
3. The UW–CIMSS AQC scheme
4. Previous validation results
a. Statistical evaluation against collocated rawinsondes
The value of the two different quality control schemes has been demonstrated in several papers (e.g., Velden et al. 1997; Holmlund 1998) as a tool to remove vectors with gross errors. Another important aspect is the extent to which the two quality indicators (RFF and QI) are related to real errors of the satellite-derived AMVs. The quality assessments are traditionally based on statistical comparisons against collocated rawinsondes and the most widely used parameter is the rms of the vector difference. Table 3 presents an example set of operational quarterly statistics derived by the respective AMV extraction centers [NOAA/NESDIS for Geostationary Operational Environmental Satellites (GOES) and EUMETSAT for Meteosat] for monitoring by the World Meteorological Organization (WMO) Coordination Group for Meteorological Satellites (CGMS). The maximum collocation distance is 150 km, the observation times have to be within 1.5 h of each other, and the maximum pressure difference is 25 hPa. The presented statistics are based on the operational AMVs derived at each extraction center.
At EUMETSAT the nrms is routinely utilized to monitor the quality of the AMV fields. Results of applying Eq. (4) to the CGMS statistics is also shown in Table 3. By this measure, it can be concluded that the final quality of the disseminated winds by the two independent AQC methods is about equal.
Figure 2 shows the verification of the EUMETSAT high-level water vapor cloud tracked winds for different mean QI based on 3718 collocated rawinsondes. The indicated error bars are based on neglecting the offset (b) in the linear dependency of Eq. (3). The offset has been estimated by performing a linear regression analysis between rms and mean wind speed using all winds in the collocation period. Also the fit of a linear regression is shown. It should be noted that the quality control scheme has been tuned on previous datasets and the statistics are based on completely independent data.
The results presented in Fig. 2 indicate a near-linear relationship between the nrms and the mean QI, indicating that the QI is a good estimator of quality, where quality is defined as being in good agreement with the point measurements provided by rawinsondes. Currently, the EUMETSAT QIs are operationally disseminated together with the AMVs. [It should be noted that inclusion of the quality information is only possible in the WMO binary unified form for the representation of meteorological data (BUFR) and not in the WMO reports of satellite observations (SATOB).] Only vectors with a QI above 0.3 are disseminated.
A similar look at the behavior and performance of the UW–CIMSS AQC algorithm as applied operationally by NOAA/NESDIS on GOES vector fields is shown in Table 4. Statistics derived over the U.S. rawinsonde network in July 1995 for five different cases are presented (Nieman et al. 1997). In this case a monotonic relationship between the RFF and the nrms is evident.
Based on the above analysis, the nrms is a reliable indicator of true quality and is therefore also utilized in the following discussion.
b. Statistics against model analyses
The automatic quality control schemes have also been evaluated against NWP fields by the respective operational centers. The EUMETSAT satellite-derived datasets are routinely monitored at ECMWF (Rohn et al. 1998a). Figure 3 presents monitoring statistics against ECMWF analyses for low-level visible and high-level WV winds. Following the practice of operational monitoring against collocated rawinsondes (Schmetz et al. 1993), observations deviating by more than 30 m s−1 in speed or 60° in direction are excluded. The statistics for high-level (<400 hPa) infrared winds are very similar to the high-level WV winds and are therefore not presented. The relationship between the nrms and the QI for the high-level water vapor cloud tracked winds is monotonic to within the accuracy of the nrms. For the low-level winds (>700 hPa) the relationship is less clear with a minor deviation around QI 0.6. This is most likely to be a result of the fact that the QI has mainly been tuned with WV cloud-tracked winds (Holmlund 1998) and that the characteristics of the low-level winds are likely to be slightly different than for high-level winds. However, for the current utilization of the QI as a tool to select the most useful subset of vectors (of the total wind field) for assimilation, this is not of major impact as all accepted vectors are treated in the assimilation with a similar weight. The only situation where the absolute value of the QI plays an additional role is in the selection of the best wind in an analysis grid box for assimilation (see section 6a). In this case all vectors within a grid box are affected by almost the same atmospheric conditions and again the impact on the selection process is expected to be minor. A possible future development is to use the QI to modify the observation operator in the assimilation scheme. In such an approach the impact of the tuning process has to be carefully assessed.
Similar comparisons are performed for GOES winds using the UW–CIMSS AQC scheme against National Centers for Environmental Prediction (NCEP) Aviation model analyses (AVN). Figure 4 shows an example plot between RFF and vector rms error, again indicating the nearly linear relationship that exists. This supports the rawinsonde comparisons in that the RFF and QI are good indicators of vector quality, and could be used as data selection tools in data assimilation.
5. Comparison of the EUMETSAT and UW–CIMSS AQC schemes
a. Subjective analysis
The previous section showed validation results based on winds derived from Meteosat data for the EUMETSAT QI scheme and from GOES data for the RFF based scheme. In order to help identify the strengths and weaknesses of the two AQC schemes, the EUMETSAT QI scheme was adapted to work on GOES and Geostationary Meteorological Satellite (GMS) data. For extensive comparisons and impact studies, high-density wind fields were derived from GOES and GMS images over the North Pacific region using the UW–CIMSS winds algorithm during the NORPEX (Langland et al. 1999) field experiment. The vector fields were independently subjected to both the RFF and QI based AQC schemes.
Figure 5 presents a high-level wind field over the northern Pacific Ocean for which vectors accepted only by the RFF (RFF > 50) are presented in blue and the winds accepted only by the QI (QI ≥ 0.6) scheme are presented in red. For this subjective analysis the QI value 0.6 was chosen as a compromise between quality and coverage. This value was also selected as the threshold for the combined AQC approach discussed later in section 6. Winds accepted by both schemes are in green. Only winds faster than 10 m s−1 are presented. Figure 6 presents the low-level vector field derived from visible imagery with the same color coding. The vectors accepted by both AQC schemes depict the main features of the flow (most coherent and consistent flow). In the jet stream area in Fig. 5 (middle section of the left portion of the picture indicated by the rectangular box), the QI scheme retains relatively more winds (even those that show large accelerations in the vector components;low scores in the speed consistency test). This is a result of the QI approach, which derives a quality indicator that depends on many different tests. Vectors are not edited based solely on a poor consistency test. The RFF based scheme tends to retain more winds in the slow and also higher curvature regions (e.g., in the anticyclonic region in Fig. 6).
b. Validation against NCEP forecasts
Table 5 presents the nrms for one wind field derived during NORPEX against the background field used in the winds processing (NCEP aviation model 12-h forecast) for winds at three layers and three channels, divided into four categories. The four categories are based on the QI and RFF values provided by the two AQC schemes. The thresholds for the categories have been chosen according to the previous findings for the QI (winds with QI > 0.60 are considered good) and the operational procedure utilizing the RFF (winds with RFF > 50 are accepted). The main conclusion from Table 5 is that, in general, the vectors that both schemes find to be good (QI > 0.60, RFF > 50) are indeed of high quality (as compared to the background field), whereas when both schemes rate the quality as poor (QI < 0.60, RFF < 50), the nrms is much higher. This is an expected result. The results are more interesting when the two schemes disagree. The statistics tend to suggest that the QI scheme is capable of retaining more good winds than the RFF-based scheme, given the current thresholds (e.g., the 935 high-level WV wind vectors accepted by the QI scheme, and rejected by the RFF scheme, have a nrms of 0.41 that is very close to the good winds with an nrms of 0.39 accepted by both schemes). However, the winds accepted by the RFF and rejected by the QI scheme seem better than those rejected by both schemes. This generally indicates that both schemes tend to have some skill in identifying good winds. It should also be noted that for low-level IR winds, the RFF performs better than the QI scheme (the winds accepted by the RFF and rejected by the QI scheme have a nrms of 0.51, whereas the winds accepted by the QI scheme and rejected by the RFF scheme have a nrms of 1.48). This verifies the earlier findings versus ECMWF statistics.
c. Validation using the ECMWF assimilation scheme
Based on the previous validation experience with the two systems, combinations of the QI and the RFF schemes were explored. As a first step, the results of which are being reported in this study, the employment of QI values as a prefilter to the RFF scheme was investigated. Three experiments were undertaken using different QI threshold values to select which vectors (only those above the chosen value) would be subjected to the RFF scheme. The chosen QI threshold values were 0.3, 0.6, and 0.9. These thresholds were chosen because 0.3 represents the operational cutoff, 0.6 still provides good coverage and reasonable quality, and 0.9 retains only the very high-quality winds. In addition to utilizing the QI as a prefilter to the RFF scheme, it was also used to select the best vector for each assimilation box in the model experiments outlined in section 6a. The experiments suggested that the most promising approach was to use the QI with threshold 0.6 to prefilter the raw wind field before submitting to the RFF scheme. The combined approach with a QI threshold set to 0.3 did not indicate a strong impact because only a small number of poor vectors was removed, and these the RFF scheme alone handles well. The QI threshold of 0.9 did not keep enough vectors for the RFF scheme to perform a coherent analysis of the data. Therefore, only the experiment results for the QI threshold of 0.6 are discussed in more detail.
GOES and GMS winds were derived between 1 and 7 February 1998 during NORPEX by UW–CIMSS and quality controlled by the RFF-based approach only (RFF). This week’s worth of datasets were then rederived using the combined AQC scheme with the QI information (0.6 threshold) as a prefilter to the RFF scheme (RFFQI60). Both datasets were disseminated to ECMWF. The AMVs are compared to the wind fields of the ECMWF 6-h operational forecast with 31 model levels and T319 spectral resolution (Courtier et al. 1998;Rabier et al. 1998). For the statistical evaluation the background field was interpolated to a latitude–longitude grid with 1.5° resolution. The model winds were then interpolated to the location and pressure of the satellite wind observations.
For evaluation, the wind vectors before and after the combined AQC approach were sorted into classes based on their quality estimates. The mean wind speed, rms departure, and bias against the model-analyzed winds were computed separately for each quality class. The relation between the departures of the satellite winds from the background field and the assigned quality is used to assess the information content of the different quality estimates of RFF and QI. The RFF and RFFQI60 experiments are shown for high-level (above 400 hPa) IR winds in Figs. 7–10. The rms background departures, speed bias, and the mean background speed are marked by solid, dotted, and dashed lines, respectively, and the number of observations in each quality class is indicated by the histogram. Furthermore, the evaluation includes partitioning of observations in the Northern Hemispheric extratropics (20° < latitude < 90°, Figs. 7 and 8).
The plot in Fig. 7 reveals decreasing rms departure with increasing RFF quality estimates with a simultaneous decrease in the mean wind speed and the speed bias. Also, the nrms (not shown) is generally decreasing from a value around 0.27 at RFF equals 50, to 0.23 at RFF equals 80, and 0.21 in the highest RFF class with more than 50 observations. The relation between background departures and the quality estimate assigned by the QI/RFF combination scheme is presented in Fig. 8. The rms departure from the background as well as the negative bias show again a decrease with increasing quality estimate QI, similar to the RFF postprocessed winds. However, for winds in quality classes above QI = 0.8, the mean wind speed is increasing. This is also reflected in a better behavior of the nrms (not shown) that is now decreasing from 0.37 at quality mark 0.6 to 0.28 at quality mark 0.9 with a minimum in the highest quality class of 0.16, indicating that the combined scheme is able to extract more consistent winds than the RFF only scheme for the highest quality class.
In the tropical region the relationship between the quality estimate and the background departure appears to be less well behaved. The rms background departures and quality indicators are plotted for high-level IR winds in Figs. 9 and 10. Please note the different scale of the departure (y axis). The RFF data (Fig. 9) show both decreasing rms departures and bias with increasing quality mark while the mean wind speed is almost constant. The RFFQI60 winds (Fig. 10) reveal a pronounced step around QI = 0.85, which is likely related to one of the vector consistency tests indicating the need of further tuning for the tropical region. This step has been also observed in the real-time operational monitoring of the Meteosat winds (Rohn et al. 1998b). Below this threshold the rms departures are large and the mean wind speed low. Above this threshold the bias and the rms departures decrease rapidly with an increase in the mean wind speed. This finding indicates that the 0.6 threshold may not be optimal for all channels. The impact of this will be addressed in the next section. The behavior of the nrms for these cases is similar to the behavior for nrms for the extratropical high-level IR winds. For the RFF only case the nrms is 0.94, 0.56, and 0.65 for RFF values 50, 80, and 83 (which still contains more than 50 vectors), respectively. For the combined scheme the nrms is 0.67, 0.38, and 0.18 for quality marks 0.6, 0.9, and 1.0, respectively, again indicating a better behavior of the combined scheme.
The evaluation of the other levels and other channels are to a large extent similar to those in Figs. 7–10 and are therefore not discussed in detail. Figures 11 and 12 present the results for the low-level (below 700 hPa) visible winds in the Tropics as an example. For the visible winds the nrms (not shown) for the RFF only scheme behaves better than for the IR high-level winds with nrms values of 1.18, 0.74, and 0.46 at RFF values 50, 80, and 97, respectively. For the combined scheme the nrms values are correspondingly 0.75, 0.46, and 0.41 for quality marks 0.6, 0.9, and 0.98.
In summary, both schemes tend to decrease the rms error at all levels over both areas (tropical and extratropical). The RFFQI60 scheme tends to have a neutral impact on the mean speed bias of all accepted low- and midlevel vectors, but decreases the bias of the high-level vectors especially for the winds in the respective highest quality categories. The RFF approach also generally results in a neutral impact on the speed bias of the low- and midlevel vectors, whereas for the high-level vectors in the extratropics the bias is slightly increased and in the tropical region decreased. The biggest difference between the impact of the two schemes is related to the mean speed of the retained wind vectors, which is generally increased for high QI. This is most likely an effect of the QI scheme taking the wind speed into account in the different tests (see Table 1).
6. Forecast impact studies with the ECMWF model
a. Assimilation strategy
For our study, the operational version of the ECMWF 4DVAR with an analysis resolution and assimilation strategy identical to the configuration outlined in section 5c is employed for examining dataset impact on forecasts (Klinker et al. 1999). The first experiments, using the originally produced high-density winds during NORPEX, follow the operational usage of winds at NOAA/NESDIS (i.e., only winds with a quality estimate of RFF > 50 were considered). Selection of vectors for assimilation was simply determined by the closest vector to each analysis grid point, but still within 1.25° latitude and longitude. This follows the current practice by the ECMWF operational assimilation of cloud drift winds from the IR and WV channels. In addition to cloud drift winds, winds from cloud-free regions were produced from the WV imagery data (Velden et al. 1997). These “clear-sky” water vapor winds were activated for the assimilation experiments. During NORPEX, advanced processing of low-level cloud drift winds from the VIS channel were introduced and also included into the datasets for assimilation. The summary of the data selection is presented in Table 6.
For the experiments using the QI-values, all winds in the Northern Hemisphere extratropics with a QI > 0.60 were considered. In the tropical belt a more restrictive use was introduced based on the findings in Fig. 10, with a minimum QI value of 0.85 for high-level vectors (Table 7). This is similar to the approach for the current operational usage of Meteosat winds that is currently under investigation (Rohn et al. 1998b).
To test the impact of the combined AQC method on NWP, the NORPEX high-density winds were assimilated into the ECWMF system according to Tables 6 and 7 during the two-week period from 25 January to 7 February 1998. Early experiments with the high-density satellite data showed some problems in assimilating large volumes; these are likely related to horizontally correlated errors in the observation data and by overwhelming other observations. Therefore the high-density wind data needed to be thinned prior to assimilation by selecting one vector per analysis gridpoint location. For each case the vector with the highest quality (highest QI value) within a selection box was chosen for assimilation, resulting in a minimum horizontal distance between assimilated observations of 1.25° as shown by Järvinen and Undėn (1997). In the vertical, only one satellite wind was allowed per nearest model pressure level (50, 70, 100, 150, 200, 250, 300, 400, 500, 700, 850, 925, 1000 hPa).
b. Forecast impact
The main objective of the NORPEX campaign was to test adaptive observing strategies in order to improve the forecast skill over the United States (Langland et al. 1999). We follow the same evaluation methodology used by previous investigators to assess the combined impact of both high-density satellite winds and dropsondes on NWP forecasts (Szunyogh et al. 1999; Langland et al. 1999). In agreement with these studies, the verification is restricted to a western North American region (30° < latitude < 60°; −130° < longitude < −100°), and we consider the rms errors of 48-h forecasts for both the 1000- and 500-hPa geopotential surfaces. The results are summarized in Table 8.
In order to show the added value of the combined AQC scheme, Table 8 presents the experiments run with datasets that were quality controlled by each of the respective methods alone. The QI-only experiment (QI > 0.6) did not employ the RFF-based scheme at any point. Both experiments that used the RFF scheme (RFF only, RFFQI60) led to reduced forecast errors, whereas the QI-only scheme actually increased the errors slightly. It is encouraging to see that by utilizing the combined AQC approach, the most notable positive forecast improvements (8%–12%) are achieved and that the combined RFFQI60 shows an improvement of 4% (i.e., roughly a third of the total) with respect to the RFF-only scheme. This can be explained by the fact that the QI scheme is able to provide a more consistent dataset for the RFF scheme, which is then able to perform a mere coherent analysis and quality control of the data. It is also important to notice that the experiments also utilized low-level VIS winds and clear-sky water vapor winds that were not used by the control run. The impact of these additional vectors in the operational setup is, however, limited as the existing winds (IR and WV cloud-tracked winds) already provide a good coverage at the resolution of the assimilation scheme. Therefore the main impact in the experiments comes from the improved data selection.
The observed forecast improvements are on the order of those found by NCEP during NORPEX by the added dropsonde observations over the Pacific Ocean (Szunyogh et al. 1999). And presumably, the 15%–20% reduction in forecast error found by Langland et al. (1999) with the navy global model using the RFF-only satellite wind datasets would benefit from the improved AQC method outlined here to yield even more impressive results.
The restriction to a particularly limited verifying region and a single forecast time requires caution especially regarding the relatively short experimentation period of two weeks. To examine this, the anomaly correlation of the forecasts for two geopotential fields (1000 hPa, top panel, and 500 hPa, bottom panel) over the entire North American region are shown in Fig. 13. The region is bounded by 25°–60°N and 75°–120°W. The anomaly correlation of the geopotential at 1000 and 500 hPa both show that the individual quality control schemes (RFF and QI only) both have a relatively neutral impact up to about forecast day 4. Beyond day 4, both schemes have a small positive impact at 500 hPa. The combined AQC (RFFQI60) shows mainly a neutral impact through forecast day 3, but a notable improvement beyond that adding up to as much as 24 h to the prediction skill.
The high-density wind data produced at UW–CIMSS for NORPEX covered a large part of the Northern Pacific. Hence it can be expected that the largest impact of such a dataset should be found over North America and that the improvements would be smaller on a hemispheric scale. An example of the geographical distribution of short-term forecast error is shown in Fig. 14. Both panels show the differences in the averaged rms error (14 cases) of the geopotential forecast at 250 hPa between the system initialized with the RFFQI60 data and the control. The forecast field is verified against its own analyses. The 48-h forecast error differences (top panel) reveal an extended area of reduced errors (in light gray) over the North Pacific Ocean. In the 72-h forecast (bottom panel) the positive forecast impact has moved east showing propagation into the NORPEX target area (western United States and Canada).
7. Conclusions
The current operational automatic quality control schemes for satellite-derived atmospheric wind vector fields at high densities as applied at NOAA/NESDIS (UW–CIMSS scheme) and EUMETSAT have been briefly summarized. Even though the two schemes differ in their respective approaches, they generally classify the quality of vectors in a similar fashion. Both schemes have been shown to have advantages and disadvantages. The NOAA/NESDIS UW–CIMSS scheme is capable of providing more coherent wind fields and adjusts for some height assignment problems. The EUMETSAT scheme is capable of retaining more winds in the fast flow regimes and lends itself more easily for analysis and interpretation due to its straightforward formulation. It has been shown that the best vectors are generally those accepted by both schemes, whereas winds rejected by the two schemes simultaneously have a low reliability as single-level point measurements. In the case of disagreement between the two schemes, the EUMETSAT quality indicator (QI) scheme is capable of identifying and retaining more high-level water vapor and low-level visible winds, whereas the NOAA/NESDIS UW–CIMSS recursive filter function (RFF) based scheme retains a better low-level IR vector field. Detailed analysis of the NORPEX cloud and water vapor drift wind sets by comparison with the ECMWF 6-hour forecasts revealed insight into the information content of the two different automatic quality control estimates. The QI and RFF values indicate the potential to mark the quality of single observations with respect to rms background departure and speed bias for data assimilation purposes.
Numerical model impact experiments were performed on a two-week dataset centered on the North Pacific Ocean during the NORPEX field program in 1998 to assess the impact of the quality control schemes on forecasts. The first two experiments investigated the use of the individual quality control methods alone (RFF only and QI only), and the third assimilated a revised dataset based on the application of the QI filter prior to the RFF-based scheme and using the quality indicators to select the final winds for input (RFFQI60 experiment). The most promising approach was to use the QI with threshold 0.6 to prefilter the raw wind field before submitting to the RFF scheme, and to subsequently use the QI, especially in areas with a high vector density, to select the best of the retained winds for data assimilation into the forecast model.
In general, the results from the assimilation and forecast impact studies show the experimental GOES and GMS high-density cloud and water vapor drift winds provided over the North Pacific region during NORPEX lead to a positive impact on the ECMWF model two-day forecast of the 1000-hPa geopotential height field over a western North America region, which was used as the primary verifying area for the NORPEX campaign. Most notably, the combined AQC approach leads to an 8%–12% reduction in rms forecast errors versus control forecasts that only assimilated operationally available observations. This result is consistent with the positive experience of using the NORPEX satellite-derived dataset in other assimilation and forecast systems (Szunyogh et al. 1999; Langland et al. 1999). The experimental AQC strategy also reveals a positive impact on the medium-range ECMWF forecasts over North America for this period as verified against observations.
This study yields strong evidence for the use of the combined QI and RFF quality control methods for high-density satellite-derived wind fields. It also suggests this strategy can yield good information on the quality of the retained vector field, and the resulting AQC flag values can be employed within the screening decisions of the ECMWF (and other) data assimilation system. The combined AQC strategy is being implemented into the satellite data processing centers at NOAA/NESDIS and EUMETSAT, and the AQC values will be made available to the user community. Future work will concentrate on further optimizing the combined AQC approach described initially in this study.
Acknowledgments
The work of M. Rohn was conducted under the EUMETSAT fellowship program at ECMWF. The UW–CIMSS efforts were supported under NOAA Grant NA67EC0100. The authors would like to thank Steve Wanzong and David Stettner of UW–CIMSS, and Jaime Daniels of NOAA/NESDIS for their contributions.
REFERENCES
Bhatia, R. C., P. N. Khanna, and S. Prasad, 1996: Improvements in Automated Cloud Motion Vectors (CMWs) derivation scheme using INSAT VHRR data. Proc. Third Int. Winds Workshop, Ascona, Switzerland, EUMETSAT, 37–43.
Courtier, P., E. Andersson, W. Heckley, J. Pailleux, D. Vasiljevic, M. Hamrud, A. Hollingsworth, F. Rabier, and M. Fisher, 1998: The ECMWF implementation of three-dimensional variational assimilation (3D-Var). Part 1: Formulation. Quart. J. Roy. Meteor. Soc.,124, 1783–1807.
Hayden, C. M., and R. J. Purser, 1995: Recursive filter objective analysis of meteorological fields: Applications to NESDIS operational processing. J. Appl. Meteor.,34, 3–15.
Holmlund, K., 1998: The utilization of statistical properties of satellite-derived atmospheric motion vectors to derive quality indicators. Wea. Forecasting,13, 1093–1104.
Järvinen, H., and P. Undén, 1997: Observation screening and background quality control in the ECMWF 4D-Var data assimilation system. ECMWF Research Dept. Tech. Memo. 236, 33 pp. [Available from ECMWF Librarian, Reading, Berkshire RG2 9AX, United Kingdom.].
Kelly, G. A., 1993: Numerical experiments using cloud motions winds at ECMWF. Proc. Second Int. Winds Workshop, Tokyo, Japan EUMETSAT, 227–244.
Klinker, E., F. Rabier, G. Kelly, and J.-F. Mahfouf, 1999:The ECMWF operational implementation of four-dimensional variational assimilation. Part III: Experimental results and diagnostics with operational configuration. ECMWF Research Dept. Tech. Memo 273, 27 pp. [Available from ECMWF Librarian, Reading, Berkshire RG2 9AX, United Kingdom.].
Langland, R. H., and Coauthors, 1999: The North Pacific Experiments (NORPEX-98): Targeted observations for improved North American weather forecasts. Bull. Amer. Meteor. Soc.,80, 1363–1384.
Le Marshall, J., N. Pescod, B. Seaman, G. Mills, and P. Stewart, 1994: An operational system for generating cloud drift winds in the Australian region and their impact on numerical weather prediction. Wea. Forecasting,9, 361–370.
Nieman, S. J., W. P. Menzel, C. M. Hayden, D. Gray, S. T. Wanzong, C. S. Velden, and J. Daniels, 1997: Fully automated cloud-drift winds in NESDIS operations. Bull. Amer. Meteor. Soc.,78, 1121–1133.
Rabier, F., A. McNally, E. Andersson, P. Courtier, P. Undén, J. Eyre, A. Hollingsworth, and F. Bouttier, 1998: The ECMWF implementation of three dimensional variational assimilation (3D-Var). Part II: Structure functions. Quart. J. Roy. Meteor. Soc.,124, 1809–1830.
Rohn, M., G. Kelly, and R. W. Saunders, 1998a: Experiments with atmospheric motion vectors at ECMWF. Proc. Fourth Int. Wind Workshop, Saanenmöser, Switzerland, EUMETSAT, 139–146.
——, ——, and ——, 1998b: Use and impact of atmospheric motion vectors at ECMWF. Proc. 9th Conf. on Satellite Meteorology and Oceanography, Paris, France, Amer. Meteor. Soc., Société Météor. de France, and EUMETSAT, 360–363.
Schmetz, J., K. Holmlund, J. Hoffman, B. Strauss, B. Mason, V. Gärtner, A. Koch, and L. van de Berg, 1993: Operational cloud-motion winds from Meteosat infrared images. J. Appl. Meteor.,32, 1206–1225.
Szunyogh, I., Z. Toth, S. Majumdar, R. Morss, C. Bishop, and S. Lord, 1999: Ensemble-based targeted observations during NORPEX. Preprints, Third Symp. on Integrated Observing Systems, Dallas, TX, Amer. Meteor. Soc., 74–77.
Tokuno, M., 1996: Operational system for extracting cloud motion and water vapour motion winds from GMS-5 image data. Proc. Third Int. Winds Workshop, Ascona, Switzerland, EUMETSAT, 61–68.
Tomassini, C., 1981: Objective analysis of cloud fields. Proc. Satellite Meteorology of the Mediterranean, Erice, Italy, ESA, SP-159, 73–78.
Velden, C. S., 1996: Winds derived from geostationary satellite moisture channel observations: Applications and impact on numerical weather prediction. Meteor. Atmos. Phys.60, 37–46.
——, C. M. Hayden, M. S. J. Nieman, W. P. Menzel, S. Wanzong, and J. S. Goers, 1997: Upper-Tropospheric winds derived from geostationary satellite water vapor observations. Bull. Amer. Meteor. Soc.,78, 173–195.
——, T. L. Olander, and S. Wanzong, 1998: The impact of multispectral GOES-8 wind information on Atlantic tropical cyclone track forecasts in 1995. Part I: Dataset methodology, description, and case analysis. Mon. Wea. Rev.,126, 1202–1218.
The observed frequency (in a logarithmic scale) vs QI class for the five different consistency tests (speed = black, direction = red, vector = brown, local = blue, and forecast = green) employed in the EUMETSAT AQC scheme derived 0900 GMT 7 Feb 1988
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
Mean QI vs nrms derived from 3718 rawinsondes collocated with high-level water vapor winds derived by EUMETSAT during Jul–Sep 1998
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
Statistics for (a) low-level visible and (b) low-level IR as well as (c) high-level WV winds vs ECMWF analyses for Oct 1997. Solid line = normalized rms, dashed line = normalized standard deviation, histogram = number of winds per quality bin
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
Statistical comparison between GOES winds RFF and vector rms error vs NCEP AVN model analyses for a period in Sep 1998. Results are based on several thousand comparisons and are presented in m s−1
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
GOES water vapor winds for 2330 GMT 3 Feb 1998 overlaid on the corresponding water vapor image. Green vectors represent winds accepted by both the RFF and the QI schemes, the blue vectors represent winds retained by the RFF scheme only, and the red vectors are retained by the QI scheme only. The QI acceptance threshold in this case is 0.6, and the RFF threshold is 50
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
GOES low-level visible channel winds for 2330 GMT 3 Feb 1998 overlaid on the corresponding visible image. The color coding is the same as in Fig. 5
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
Comparison of background departures of high-level (above 400 hPa) extratropical IR winds from the original NORPEX dataset (RFF only) to assigned RFF quality estimates. (The line for rms is solid, that for speed bias dotted, and that for mean background speed dashed.)
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
Comparison of background departures of high-level (above 400 hPa) extratropical IR winds from the revised NORPEX dataset (RFFQI60) to assigned QI quality estimates. (The line for rms is solid, that for speed bias dotted, and that for mean background speed dashed.)
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
As in Fig. 7, but now for the tropical high-level (above 400 hPa) IR winds
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
As in Fig. 8, but now for the tropical high-level (above 400 hPa) IR winds
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
As in Fig. 7, but now for the low-level (below 700 hPa) extratropical VIS
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
As in Fig. 8, but now for the low-level (below 700 hPa) extratropical VIS
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
Anomaly correlations of the geopotential height field forecasts over North America at 1000 (top) and 500 hPa (bottom) for 14 cases (15 Jan–7 Feb 1998). The forecasts are from the ECMWF model and verified against rawinsonde observations
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
Differences of rms forecast error for the geopotential at 250 hPa between the RFFQI60 experiment and the control. Top: 48-h forecast, bottom: 72-h forecast. Negative values indicate reduced forecast errors and are marked in light gray shading, whereas positive values are marked dark. All other areas do not show a significant change and are clear
Citation: Monthly Weather Review 129, 3; 10.1175/1520-0493(2001)129<0517:EAQCAT>2.0.CO;2
The EUMETSAT AMV consistency tests. Di (x, y), Vi (x,y), and Si (x, y) are the direction, speed, and vector derived from the first image pair (i = 1) or the second image pair (i = 2) of a sequence of three images. F(x, y) is the forecast vector interpolated to thelocation of the target (x, y) and (x − i, y − j ) refers to the surrounding locations. The weight of the individual tests in the derivation of the final quality is given by w and the exponent of the normalization function by a
The rms vector difference normalized by the mean wind speed (nrms) for low-level winds (below 700 hPa) in different speed classes (in m s−1) derived against collocated rawinsondes during Sep–Oct 1998. Number of collocations (Noc) is also indicated
CGMS statistics for rms vector difference, speed bias, mean rawinsonde speed, and nrms for summer 1998 for operational GOES-8 and Meteosat high-level (pressure less than 400 hPa) water vapor winds. The statistics are based on several thousand collocated rawinsondes (R/S; maximum distance between observations is 150 km). All units are m s−1 except for nrms, which is nondimensional
Rawinsonde collocation statistics [rms vector difference, speed bias, rawinsonde mean speed, and the rms vector difference normalized with the rawinsonde mean speed (nrms)] for different RFF classes for several hundred collocations derived during five days in Jul 1995 over the United States. The collocation radius was 1.5° and the results are in m s−1 except for nrms, which is nondimensional
Nrms for AMVs vs NCEP AVN 12-h forecast at different levels, high = H (above 400 hPa), mid = M (400–700 hPa), and low = L (below 700 hPa), for three channels (IR, WV, and VIS), and for different combinations of RFF and QI. The number of collocations is in brackets for each case
Data selection for the first (original dataset) experiment. Low (below 700 hPa), mid (400–700 hPa), and high (above 400 hPa)
Data selection according to quality estimate QI (RFFQI60 experiment). Low (below 700 hPa), mid (400–700 hPa), and high (above 400 hPa)
Influence of the experimental assimilation of high-density winds on ECMWF model 48-h forecast errors of the 1000- and 500-hPa geopotential surfaces. The relative forecast impact is given as rms error differences between forecasts starting from the analysis using the advanced multispectral (but experimental) satellite winds derived by UW–CIMSS for NORPEX, and control analyses. The control analyses used all operationally available data including limited satellite wind fields produced by NOAA/NESDIS from GOES-9, and GMS winds over the western North Pacific produced by the Japanese Meteorological Agency (no low-level VIS or clear-sky water vapor winds were assimilated). Results are shown for the RFF-only, QI-only, and the combined RFFQI60 experiments. The evaluation period is 25 Jan to 7 Feb 1998. Both the absolute forecast rms errors and the difference values are averaged over the region of interest (30° < latitude < 60°; −130° < longitude < −100°). Negative difference values indicate forecast improvement over the control. The forecasts are verified against their own analyses
