Aircraft Meteorological Data Relay (AMDAR) weather reports are a type of high spatiotemporal data currently widely used in weather monitoring and prediction. A recent Chinese AMDAR project began in 2003 has made rapid progress. However, the assessment and accuracy of these Chinese AMDAR reports have yet to be thoroughly discussed. A comparison of temperature and wind observations between Chinese AMDAR reports and rawinsonde data between 2004 and 2010 is conducted in this paper. Results demonstrate that the root-mean-square error (RMSE) between these two sets of data is 1.40°C for temperature, 3.56 m s−1 for wind speed, and 28° for wind direction. Because of the particularity of observation and inversion method, comparison results are not only affected by AMDAR measurement and reporting error but also by spatial and temporal representativeness, flight phases, and the environment. This evaluation helps create a complete estimation of the accuracy of Chinese AMDAR in order to assist with data assimilation.
Modern commercial aircraft are equipped with meteorological sensors and associated sophisticated data acquisition and processing systems. Through the Aircraft Communications and Reporting System (ACARS) and other communication systems, these meteorological observations are relayed to the ground, forming a complete system called Aircraft Meteorological Data Relay (AMDAR). AMDAR was proposed for the First Global Atmospheric Research Program (GARP) Global Experiment (FGGE) in the 1970s. Global cooperation on AMDAR was established in 1998 by the World Meteorological Organization (WMO) AMDAR Panel (WMO 2003).
AMDAR weather reports are a reliable source of meteorological data that are commonly acknowledged and currently widely used in weather monitoring and prediction. Compared to conventional observations, the direct use of AMDAR reports to continuously monitor meteorological elements near airports is more convenient and effective (Mamrosh 1998; Mamrosh et al. 2001; Kurimski and Brusky 2006). In addition, AMDAR reports can be used to supplement conventional meteorological observations (Tenenbaum 1991, 1996; Moninger et al. 2003; Lese and Ammerman 2008).
AMDAR reports are also used in numerical weather prediction (NWP). Assimilation tests have shown that in a certain region, the AMDAR reports consistently improve numerical simulation results, regardless of the coverage density (Rukhovets et al. 1998; Pouponneau et al. 1999; Cardinali et al. 2003, 2004; Benjamin et al. 2010). To this effect, an estimation of observational error is needed, especially for the data assimilation system. Several comparison experiments have been performed that estimate data accuracy. In an earlier study, a comparison was investigated between ACARS, rawinsonde, cloud motion, and the Visible and Infrared Spin Scan Radiometer (VISSR) Atmospheric Sounder (VAS) within a spatial and temporal window (111 km, 120 min).1 Results indicated that ACARS reports provide independent data sources that complement other wind data for constructing wind field analyses (Lord et al. 1984). Schwartz and Benjamin (1995) compared temperature and wind observations between the ACARS reports and rawinsonde data surrounding Denver, Colorado, and suggested that the standard deviation was 0.59 K for temperature and 2.84 m s−1 for wind speed within a window of 25 km and 15 min. Also, a collocation study of ACARS reports with other ACARS reports in a very small window of 1.25 km and 2 min was performed to estimate the wind and temperature observational errors for ACARS alone, where the RMSE was found to be 0.69–1.09 K for temperature and 1.6–2.5 m s−1 for wind speed (Benjamin et al. 1999).
The Chinese AMDAR program commenced late 2003. Since 2004, a part of the Chinese AMDAR reports have been shared worldwide via the Global Telecommunication System (GTS). The AMDAR reports collected in China have not been extensively used yet, especially in the studies related to data assimilation, which can be attributed to the shortage of a systematic evaluation of the data quality in a quantitative way. Basic research on data quality has shown a 0.3% error rate for temperature observations and 1.3% for wind observations (Liao and Xiong 2010), but there is hardly any existing work that makes exact estimations of the accuracy of Chinese AMDAR reports. Relevant studies are urgently needed.
Previous studies (Schwartz and Benjamin 1995) have also shown that the flight phase, spatial–temporal separation, and atmospheric variability all influence the comparison results. A recent study (Gao et al. 2012) employs an intercomparison by Tropospheric Airborne Meteorological Data Reporting (TAMDAR), rawinsonde, and a 6-h forecast from a Weather Research and Forecasting (WRF) Model, and the estimation result shows that the wind speed observational error of TAMDAR is a function of the wind speed magnitude itself. The problem, however, is that flight level, ambient wind speed magnitude, and spatial separation are all closely related, and none is identifiably dominant in those studies. In this study, a variable separation approach is used to solve this problem. In addition, because it is a peculiar feature of AMDAR reports, the connection between wind observations and temperature observations will be discussed.
In this paper, the observations of temperature and wind between AMDAR and 10 rawinsonde stations are compared and analyzed in the given spatial–temporal windows. Section 2 introduces the AMDAR reports, rawinsonde data, and the method used to create matched data pairs. Section 3 details the comparison results and discusses their influencing factors. Section 4 provides a summary and conclusions.
2. Data description and experimental design
a. Chinese AMDAR reports
The original Chinese AMDAR reports used in this study are collected by 60 planes from different Chinese airlines and only the reports from 20 out of the 60 planes are internationally shared. The sampling interval time of the AMDAR system varies with time and flight phase—that is, 6 s in the first 60 s and then 35 s during the ascending phase, 180 s while in the cruise phase, and 60 s during the descending phase. These original reports are decoded and quality controlled by China’s meteorological information center. The quality control scheme is properly adjusted from the Meteorological Assimilation Data Ingest System (MADIS) automated aircraft reports quality control scheme to a Chinese version (Tao et al. 2009). The observations in this study cover the periods from 1 January 2004 to 31 December 2010 and contain these variables: time, latitude, longitude, pressure altitude, flight level, temperature, wind speed, wind direction, and flight phase (i.e., ascent, descent, cruise, and turbulence). Figures 1 and 2 show the spatial and temporal distributions of the AMDAR reports, respectively.
b. Chinese rawinsonde data
In China, rawinsondes are launched from 119 stations twice a day at 2315 and 1115 UTC. The observations are transported in real time by GTS and are quality controlled by China’s meteorological information center. The random observational error is about 0.2 K for temperature (Tao et al. 2006). Some studies show that in China, balloon-borne rawinsondes float mainly eastward, with a mean drift distance of 40 km once the balloon rises to 200 hPa (Li et al. 2010). The balloon’s rising velocity is about 6 m s−1, and it takes about 35 min to reach 200 hPa. In this study, we selected rawinsonde observations gathered from 10 rawinsonde stations: Beijing, Shenyang, Shanghai, Nanjing, Hangzhou, Anqing, Hong Kong, Nanning, Qingyuan, and Haikou (represented by black circles in Fig. 1), all densely covered by flight routes. In a later comparison, these observations are treated as true values.
c. AMDAR–rawinsonde data pairs
The situation favorable for creating AMDAR–rawinsonde data pairs is shown in Fig. 3. The detailed track information and its corresponding time are known from the rawinsonde data. If one AMDAR data record falls into the spatial and temporal window (100 km, 15 min) of the rawinsonde observation, then the rawinsonde observations are interpolated to the pressure altitude of the AMDAR report in a logarithmic pressure coordinate by using a linear Lagrangian interpolation method to create a matched AMDAR–rawinsonde data pair; it contains variables of time, location, pressure altitude, distance of separation (distance between the AMDAR report and the rawinsonde observation in the horizontal direction), flight phase of the aircraft, and a pair of temperature/wind speed/direction observations. A total of 14 119 data pairs are created.
It is an important characteristic of the AMDAR–rawinsonde data pairs that the pressure altitude of the AMDAR reports is closely related to the distance of separation, such as Fig. 4 shows. The mean pressure altitude of the AMDAR reports generally increases with distance of separation but declines around a distance of 35 km. The primary reasons are that higher pressure altitude usually corresponds to a farther distance from the airport, since there is an angle of slope between the flight path and the ground during ascent/descent (illustrated by Fig. 3), and as is known that there is always a distance between an airport and rawinsonde station, the average distance for the 10 stations we selected is about 35 km to the airport. As a result, the corresponding decline of mean pressure altitude that appears at a distance of 35 km is because of the density of reports gathered near the airports.
Figure 5 shows the relationship between the sample size of the matched data pairs and the pressure altitude/distance of separation. The sample size shows a decreasing trend above 450 hPa as the pressure altitude increases while at the same time reaching its maximum when the distance of separation is around 50 km. These results are not only associated with the flight path of the aircraft but also with the drift of the balloon and the reporting frequency of the AMDAR reports. The matched data pairs above 200 hPa are scarce, so in this study we only discuss the comparison results calculated below 200 hPa.
3. Comparison results
a. Difference distribution
As shown in Fig. 6, temperature observations range from −60° to 40°C, and the correlation coefficient between AMDAR and the rawinsonde observations is 0.99. Wind speed observations range from 0 to 80 m s−1. The correlation coefficient is 0.95 for wind speed and 0.94 for wind direction.
The frequency distribution of temperature differences (AMDAR − rawinsonde) satisfies normal distribution, with an average value (μ) of 0.17°C and a standard deviation (σ) of 1.40°C, which pass the Kolmogorov–Smirnov (K–S) test2 at a 5% significance level. The frequency distribution of wind speed differences also satisfies normal distribution with an average value of 0.33 m s−1 and a standard deviation of 3.56 m s−1. For wind direction, the standard deviation is 28° and the average value is almost equal to 0°, and this mode of frequency distribution rather than normal distribution represents the evidence that the vast majority of wind direction differences are relatively small.
The differences that fall out of the range (μ − 3σ, μ + 3σ) are regarded as gross differences. The proportions of gross difference for temperature, wind speed, and wind direction is 1.15%, 1.23%, and 2.68%, respectively.
b. Sources of differences
The main factors causing the differences between the AMDAR reports and the rawinsonde data are shown below.
1) Measurements and reporting error
For the AMDAR reports, the temperature (T) is calculated by probe temperature () and Mach number (M, derived from the Pitot tube system):
where is a coefficient calculated by the ratio of specific heats of dry air and probe recovery factor. The uncertainty of temperature observations is about 0.3°–0.44°C.
However, there is an additional −3°C by evaporation cooling when the sensor is dampened by clouds (WMO 2003). Therefore, it is rather important to eliminate the influence of cloud when doing the comparison. Since the moisture sensor is unavailable on AMDAR, the presence of cloud is determined by rawinsonde observation. Wang and Rossow (1995) developed a method to employ the relative humidity profile of rawinsonde data to estimate the cloud vertical structure, and it was successfully applied in many cases (Naud et al. 2003; Zhang et al. 2010). Considering the facts, the same method is employed in our study to determine the existence of the cloud. The corresponding RMSE of temperature whether the rawinsonde detects cloud is calculated separately in the given spatial–temporal window; Table 1 shows the results. Clearly, the small temporal and spatial window—that is, 20 km and 10 min, respectively—affects the accuracy of results when RMSE reaches its maximum in cloud (3.05°C), which in turn proves the effectiveness of the method of cloud detection. Under the circumstance of clouds, RMSE decreases with the amplified window and it changes a little when the separation is beyond 40 km. One possible reason is that the larger window includes many data pairs at higher flight levels where clouds at these levels would generally be composed of ice crystals that have little effect on the temperature sensor through evaporative cooling. Another reason is that the larger window smooths the clouds’ effect, probably due to its coarse representation of the local cloud coverage. To make sure our results are refrained from cloud effects, the collected data pairs will be removed when they satisfy these two conditions: rawinsonde detects the existence of cloud and the window is within 40 km and 15 min.
The wind measurement () is calculated by adding the velocity of the aircraft with respect to the earth () and the velocity of the air with respect to the aircraft ():
Here, is obtained by GPS, which is a highly accurate system. Thus, the accuracy of wind measurement is mainly dependent on , which is determined by the true airspeed (TAS) and obtained by the Pitot tube system:
where is a constant coefficient.
Inaccurate measurements of the Mach number or probe temperature will both lead to wind measurement error. The uncertainty of the wind speed is about 0.5 m s−1 with zero roll, pitch, yaw, and angle of the aircraft under perfect inertial platform alignment condition, considered as a TAS uncertainty. The estimated uncertainty of wind speed considering the flight altitude is 2–3 m s−1 (WMO 2003).
If there is only the probe temperature error or the representative difference, then
According to Eqs. (2)–(7), the temperature difference will lead to speed and direction difference. To verify this, we divide the matched data pairs between the pressure altitude of the surface and 800 hPa into groups based on temperature difference and calculate the RMSE of the speed and direction in each temperature difference group. Since the wind speed magnitude is closely related to pressure altitude, we only take observations within this relatively shallow range (surface–800 hPa) into consideration, regardless of the effect from the vertically varied wind speed. Figure 7 demonstrates that the RMSE of the speed and direction increases (decreases) with the increases (decreases) in temperature difference. This phenomenon recurs for the samples within other pressure altitude ranges (not shown). The relationship between temperature and wind makes AMDAR reports different from conventional meteorological observations and should be duly noted during operational forecasting and assimilation.
2) Representative difference
Difference associated with the separation of time and space is called representative difference (Lorenc 1986; Daley 1993), which should be crucially considered. Schwartz and Benjamin (1995) gave a breakdown of the ACARS–rawinsonde differences by distance and time of separation, and showed that the standard deviation of the temperature/wind speed difference reduced from 0.97 K/4.42 m s−1 to 0.59 K/2.84 m s−1 by limiting the spatial and temporal window from (150 km, 80 min) to (25 km, 15 min), but the high correlation between the distance of separation, the flight level, and the wind speed magnitude makes it difficult to distinguish which factor dominates.
Here, we divide the integrated pressure altitude range (surface–200 hPa) into seven pressure altitude ranges so that within each range, the relationship between pressure altitude/wind speed magnitude and distance of the separation is minimized. The variations of RMSE with distance of separation are shown in Table 2. For temperature and wind speed, RMSE increases with distance in the integrated pressure altitude range (surface–200 hPa), meanwhile within each range, the RMSE also has a tendency to grow over distance. For direction, the RMSE increases with distance generally in each range of (surface–850 hPa), (850–700 hPa), (580–470 hPa), and (470–380 hPa). Nevertheless, the RMSE in the integrated pressure altitude range (surface–200 hPa) shows an irregular variation with distance mainly due to the vertically varied wind speed magnitude. These results supplement the work conducted by Schwartz and Benjamin (1995), and further demonstrate the performance of the representative differences.
3) Flight phase and environment
(i) Flight phase
Each data pair contains current flight phase information: cruise, ascent, descent, and turbulence. Table 3 provides the comparison results for matched data pairs under these four flight phases. A spatial window for a distance of 40 km is used to reduce representative differences, since data pairs with the cruise phase generally have larger distances of separation than ascent and descent. Note that, turbulence is not a separate phase that can be encountered during any phase of flight, but it is treated as an individual phase and classified separately from the other phases in order to show the direct impact of an unstable aircraft platform on AMDAR observation. The results shown in Table 3 indicate that cruise phase has the smallest RMSE of each variable; in contrast, the turbulence phase has the largest RMSE due to inaccurate measurements of roll, pitch, yaw, and angles of the aircrafts. Meanwhile, the RMSE under the ascent phase is greater than that of descent; a reasonable explanation is that during ascent, the aircraft generally flies faster than descent, and the air temperature outside the aircraft is changing too rapidly for the probe to keep up with and, as a result, it leads to observation error.
(ii) Wind speed
Since wind speed magnitude is strongly associated with pressure altitude, to better demonstrate that the comparison results are dependent on wind speed, we also divide the integrated pressure altitude range (surface–200 hPa) into seven ranges to eliminate the influence of pressure altitude. Then based on the wind speed magnitude, the data pairs are divided into four 5 m s−1 interval speed ranges within each pressure altitude range. The RMSE for each speed range is calculated and shown in Table 4. The RMSE of temperature increases with amplified wind speed above 850 hPa. However, the RMSE of wind speed and direction shows a different tendency; that is, the former increases with wind speed within each pressure altitude range, while the latter shows a sharp decrease with speed.
Measurements of the Pitot system are negatively impacted if AMDAR reports show low wind speed relative to flight speed. An inaccurate Mach number leads to inaccurate observations of wind and temperature, according to Eqs. (1)–(6). This may explain why the RMSE of wind direction decreases sharply with wind speed. At the same time, the RMSE of wind speed increasing with wind speed magnitude is a typical characteristic of AMDAR wind observation (Gao et al. 2012).
(iii) Atmospheric stability
Atmospheric stability is also an important factor to take into account. Local changes of meteorological variables under unstable weather condition will affect the accuracy of comparison results. Here, we use K index (KI) to indicate atmospheric stability (George 1960):
where T is the dry-bulb temperature and is the dewpoint temperature. KI is calculated by rawinsonde observations. The equation KI < 30 corresponds to a stable weather condition, while KI > 30 often corresponds with atmospheric instability. Figure 8 shows the correlation between the RMSE of wind speed and KI. The window of 40 km and 15 min is used to reduce representative differences and in addition, we only select data pairs occurring in summer and autumn to reduce any seasonal changes in wind speed magnitude. The result shows that no matter in which speed range, the RMSE of the wind speed for KI > 30 is greater than that for KI < 30; as a result, it indicates the influence of atmospheric stability.
Chinese AMDAR reports provide temperature and wind observations with high spatial and temporal resolution over central and eastern China. A comparison of temperature and wind observations has been formed between Chinese AMDAR reports and nearby rawinsonde data, in order to evaluate the accuracy of Chinese AMDAR.
Frequency distributions show that temperature and wind speed differences satisfy normal distribution, but wind direction does not. Within the window of 100 km and 15 min, the standard deviation for temperature is 1.40°C. AMDAR temperatures are 0.17°C warmer than rawinsonde observations. The standard deviation for wind speed is 3.56 m s−1, and AMDAR wind speeds are 0.33 m s−1 greater than rawinsonde. There is almost no bias in wind direction observations between AMDAR and rawinsonde. The standard deviation of wind direction is 28°. If the window is limited to 20 km and 15 min, then the RMSE of temperature/speed/direction reduces from 1.40°C/3.56 m s−1/28° to 1.23°C/2.77 m s−1/27°, respectively. By estimating accuracy, we find the following:
The differences between AMDAR and rawinsonde mainly arise from instrument measurement and reporting errors, spatial and temporal representativeness, flight phases, and environmental factors.
The close relationship between wind observations and temperature observations is a specific characteristic of AMDAR observation due to the wind inversion algorithm. Both the Mach number error and the probe temperature error lead to inaccurate wind observations. Gross differences in temperature produce likewise extreme differences in wind observations.
The representative difference is crucial to building an accurate comparison—specifically, the RMSE of temperature and wind speed increase with the amplified window.
Flight phases and environmental conditions are essential factors affecting the accuracy of observation and comparison results. First, stability of the aircraft platform influences observation quality. Second, the quality of AMDAR temperature observations is deteriorated by the evaporation cooling of the temperature sensor when the aircraft is in a saturated environment. Third, the RMSE of speed (direction) increasing (decreasing) with wind speed magnitude is a typical characteristic of AMDAR wind observation. Finally, atmospheric stability is of great importance: under an unstable weather condition, comparison results are more likely adversely affected because of the local changes of meteorological variables.
This evaluation helps us to better understand the overall quality of Chinese AMDAR weather reports and the possible factors that influence them. Further, evaluation results will be used in a data assimilation system to test how they improve numerical weather prediction.
The authors thank Yu-xin Lin and Xiao-min Chen for their suggestions regarding grammar, and the reviewers for their comments on the manuscript. This work is primarily supported by the National Fundamental Research 973 Program of China (2013CB430100, 2009CB421502), the National Key Technology Research and Development Program of China (2009BAC51B01), and the National Research Fund for Public Welfare (GYHY200906014-01-01, GYHY201106004).
Because the comparison objects are not fully matched in the spatial and temporal representations, a certain window is chosen to limit the distance and time separation between the comparison objects. This spatial and temporal window is defined as window (distance separation in the horizontal direction, time separation), for example, window (25 km, 60 min).