An increasing proportion of global radiosonde stations provide high-resolution BUFR reports.
Since the mid–twentieth century, radiosonde data have been a staple of the global observing system. A code for the international exchange of radiosonde data was adopted in 1946 (U.S. Weather Bureau 1949) and later became FM 35 TEMP; see WMO (2015a) for the final revision. TEMP SHIP, TEMP DROP, and TEMP MOBILE are variants of the format used for ship, dropsonde, and mobile land reports. PILOT code (see the appendix for a listing of key acronyms used in this article) is used for wind-only reports; in this paper most references to TEMP also apply to PILOT. For operational use the reports are circulated in real time on the WMO GTS. TEMP and other alphanumeric codes were frozen in 2003 as part of the migration to table-driven code forms (see the “Valid and Invalid BUFR“ sidebar), mainly the binary code BUFR. A BUFR radiosonde template became operational in 2007 in parallel with TEMP. There are four “parts” of the TEMP code: part A (standard levels: 1000, 925, 850, 700, 500, 400, 300, 250, 200, 150, and 100 hPa, plus surface, tropopause, and maximum wind levels), part B (significant levels up to 100 hPa), part C (standard levels: 70, 50, 30, 20, and 10 hPa), and part D (significant levels above 100 hPa). BUFR does not have parts, but it does allow for the dissemination of much higher vertical resolution with the reporting of the time and position at each level and extra metadata.
VALID AND INVALID BUFR
TDCFs such as BUFR rely on the use of external tables that define all of the possible data values that can be reported, including their units and precision. Data messages are constructed using combinations of references to these table entries, allowing many types of reports to be exchanged using the same code form. New requirements can be met by adding or modifying entries in the official tables (coordinated by IPET-DRMM), but using the same encoder–decoder software. TDCFs are very flexible, so in practice a number of approved templates have been set up to provide some standardization.
WMO Codes and Migration to TDCF (www.wmo.int/pages/prog/www/WMOCodes.html) provides the formal definition of the codes and links to the templates and regulations, which are available online (www.wmo.int/pages/prog/www/WMOCodes/WMO306_vI2/TemplateExamples.html). An important feature is that a BUFR radiosonde message should contain all the data from an ascent (or all the data up to 100 hPa for the preliminary report); BUFR was not designed to contain a single part (A, B, C, or D) converted from TEMP. Unfortunately, reformatting single parts was/is common practice; using such reports would place additional work on NWP centers and require some modification to allow the different parts to be easily identified. This option was discussed and eventually rejected by IPET-DRMM and users in early 2015. The CBS letter of 11 June 2015 (www.wmo.int/pages/prog/www/WIS/wiswiki/tiki-view_blog_post.php?postId=171) reiterated that converted single parts do not conform to BUFR regulations and also stated that “It is preferable to use the highest resolution information available directly from radiosonde systems when compiling a report in BUFR” (the regulations themselves make no mention of the vertical resolution).
If the information from different TEMP parts is reformatted and merged together, then the resulting BUFR report can meet the coding regulations. Such reports have no scientific advantage over the TEMP reports and some users would prefer the TEMP reports to continue until such time as the producer can generate high-resolution native BUFR (an overlap of TEMP and native BUFR reports facilitates checking of the latter; an overlap of reformatted reports and native BUFR would cause problems for real-time users).
The main producers of radiosonde reports are NMSs. There are almost 200 NMSs but not all have radiosonde stations and some with limited resources struggle to maintain their radiosonde programs. About 20 manufacturers provide radiosondes; some systems have software options to produce high-resolution BUFR reports but others are not yet “BUFR ready,” partly reflecting their customer priorities. Several software companies provide programs to convert alphanumeric reports to/from BUFR; to date, these programs are more successful for surface reports than for radiosonde reports. WMO (through its secretariat and the CBS) tries to coordinate the complex migration to BUFR. The migration involves experts in radiosondes, codes, communications, and databases, as well as users such as numerical weather prediction (NWP) and data assimilation researchers, visualization specialists, weather forecasters, and climate researchers. Because of the complexity and the number of different data producers, the migration is taking longer than anticipated; according to the WMO’s timetable, TEMP reports should have ceased in November 2014. In practice, only a small proportion of alphanumeric reports have ceased so far (September 2016), but this proportion will increase over time. Some of the replacement BUFR reports are high resolution, some are equivalent to TEMP resolution, and, regrettably, many are currently unusable from the point of view of major NWP centers. These centers have been providing feedback to WMO and data producers.
All users of GTS radiosonde data should be preparing to use BUFR reports if they do not already do so; various resources to help new BUFR users are linked from the online supplemental material to this article. Note that in most cases the radiosonde reports available for climate or research purposes are simply copies of those exchanged in real time. Some field experiments have their own archives with more detail than TEMPs provide (e.g., Ciesielski et al. 2014). Access to high-resolution BUFR data with drift information would greatly simplify the collection of such field campaign datasets. There is an archive (not BUFR) from 1998 of U.S. high-resolution radiosonde data (see www.sparc-climate.org/data-center/data-access/us-radiosonde/). Hamilton and Vincent (1995) made the case for archiving high-resolution radiosonde reports for use in NWP and gravity wave research.
Until about 1980 most radiosondes involved manual computation and coding of reports (WMO 2014), so human errors occurred (e.g., in the sign of the temperature). Since then, radiosonde reports have become increasingly automated; this has generally improved report quality and also enabled the processing of higher data volumes, mainly more levels. There is no clear definition of what constitutes a high-resolution radiosonde report. Various European countries provide reports sampled every 2 s: for a 2-h ascent, this provides 3600 levels. One or two countries provide reports sampled every second (the typical radiosonde sampling frequency). Because of communication restrictions, some ASAP ships provide BUFR reports sampled every 10 s, typically at 400–600 levels; Fig. 1 shows an example with more detail in the BUFR report, especially in the jet, even with 10-s sampling. For the purposes of this paper we will regard sampling of 10 s (∼50 m) or better as high resolution. This is still a factor-of-10 more levels than for TEMP reports, which are often 50–100 levels (all parts combined). The scientific information content does not increase linearly with the number of levels, but very high resolution allows different users the option to subsample or process the data in ways that suit their applications. Very-high-resolution data can also be used for turbulence, wind shear, and gravity wave research. The data volumes involved are still very small compared to the amount of satellite data used in modern NWP systems.
TEMP (black) and BUFR (blue) representations of the wind speed profile, with a logarithmic vertical axis, from an ASAP ship during Nov 2014. TEMP standard levels are marked with red diamonds. The ECMWF background (12-h forecast) profile is shown as a dashed green line. (There was also a BUFR report with 47 levels up to 100 hPa; not shown.)
Citation: Bulletin of the American Meteorological Society 97, 11; 10.1175/BAMS-D-15-00169.1
After summarizing the advantages of BUFR representation in the next section, we then describe the current global status of the migration to BUFR and the processing performed before report transmission and afterward in NWP systems. There is also a more detailed discussion of what constitutes valid BUFR radiosonde reports followed by overall conclusions.
ADVANTAGES OF BUFR REPRESENTATION.
When TEMP code was introduced, the telecommunication speed and costs were overriding concerns, leading to space-saving measures such as 1) for temperature a precision of 0.2°C was used with the tenths digit also indicating the sign of the temperature [this can introduce a small bias during coding/decoding; see Ingleby and Edwards (2015)]; 2) height values were stored in three digits, where the leading digit has to be inferred, and at 500 hPa the precision reduces to 10 m; 3) wind direction was stored to a precision of 5°; and 4) a dewpoint depression larger than 5°C was stored to a precision of 1°C. The 500-hPa height is an important variable for forecast verification, and it is perhaps the most affected by the limitations of the TEMP code.
Figure 2 shows a scatterplot of the root mean square (rms) difference between 500-hPa geopotential height observations and the ECMWF model background for TEMP and BUFR reports. For most stations the BUFR statistics are smaller/better. The main reason for this appears to be the rounding in TEMP code: rounding to 10 m puts a lower bound of 5 m on the observation uncertainty (for small samples the rms can be slightly lower by chance, but note that the rms includes background uncertainty as well). Dirksen et al. (2014, Fig. 20) suggest an uncertainty of 3–4 m at 500 hPa for reference radiosondes.
Comparison of July 2015 500-hPa geopotential height BUFR minus background rms to TEMP minus background rms, where the background is an ECMWF 12-h forecast. This is for stations in WMO blocks 1–16 (much of Europe) with at least 15 TEMP and 15 BUFR reports; a few stations lie outside the range of the figure. As a simple quality-control measure, individual differences with magnitude over 50 were excluded.
Citation: Bulletin of the American Meteorological Society 97, 11; 10.1175/BAMS-D-15-00169.1
TEMP reports are split into four parts as mentioned above. Significant levels (turning points, reported in parts B and D) are chosen so that linear interpolation in height can reconstruct the complete measured profile within specified limits (up to ±1 K for temperature and ±15% for relative humidity in the troposphere and up to ±2 K for temperature in the stratosphere; up to ±10° for wind direction and ±5 m s−1 for wind speed). When the radiosonde BUFR regulations were drawn up, there was a desire to move toward high-vertical-resolution reporting and to simplify the reporting structure. Hence, there should be a single report containing all data from the ascent; for timeliness there is also a preliminary report when the radiosonde reaches 100 hPa. The preliminary reports can be very important for regional NWP systems, which operate with a very tight production schedule.
The migration to BUFR also applies to surface and aircraft reports but for these the basic content and structure of BUFR reports are much the same as for the alphanumeric forms, with BUFR containing some extra metadata. Thus, reformatting SYNOP (surface land) reports to BUFR works reasonably well. However, reformatting TEMP reports to BUFR is more difficult because of the requirement to merge different parts and questions of what to do if some parts are delayed.
CURRENT STATUS OF MIGRATION TO BUFR.
Two types of radiosonde reports in BUFR format are being exchanged globally. BUFR reports generated directly from the radiosonde data (after some processing; next section) are referred to as native BUFR reports. These native BUFR reports are often high resolution. BUFR reports generated through the conversion of TEMP reports are referred to as reformatted TEMP, or simply reformatted reports. Reformatted reports are mostly still in four parts (corresponding to TEMP parts A–D); such reports do not meet the BUFR coding regulations and are difficult for NWP centers to use (see the “Valid and Invalid BUFR“ sidebar). Reformatted reports cannot take advantage of the higher precision and other advantages of BUFR.
Figure 3 shows that as of December 2015, ECMWF received BUFR reports (both native and reformatted) from about 70% of the approximately 820 radiosonde stations worldwide; only the presence of a report was considered, regardless of report quality. Some ships have provided only BUFR since November 2014. Spanish TEMP transmissions ceased in January 2016 and others may stop in the near future (several countries have stopped sending SYNOPs). A total of 266 stations (light blue in Fig. 3) provided fewer ascents in BUFR than in TEMP code and (not shown) there are some cases where only reports up to 100 hPa are available in BUFR compared to full ascents in TEMP. Some BUFR reports (shown in red) have station position discrepancies from WMO Publication 9, Volume A (www.wmo.int/pages/prog/www/ois/volume-a/vola-home.htm), often because of the incorrect conversion from degrees and minutes to decimal degrees; errors in the sign of the longitude have also occurred. Similar problems affect BUFR surface reports from land stations. Many of the station position errors in BUFR have been corrected over the last two years, but some countries have yet to make corrections. Unfortunately, both the BUFR reports and WMO Publication 9A contain some errors and sometimes reflect out-of-date station position information.
Summary of radiosonde reports for 1–31 Dec 2015 processed at ECMWF, plotted by station: white, TAC but no BUFR (gray for wind only); green, BUFR but no TAC; red, position difference; and blue, both TAC and BUFR reports (light blue, two or more 12-h windows with TAC but no BUFR). Circles show land TEMP stations, crosses show wind-only stations, and triangles show ships (including ASAPs), shown at one location per ship.
Citation: Bulletin of the American Meteorological Society 97, 11; 10.1175/BAMS-D-15-00169.1
Figure 4 shows the maximum number of levels per BUFR report, and Fig. 5 shows those stations providing valid balloon drift positions (some stations report zero drift at all levels). There is no indicator available to distinguish native BUFR from reformatted TEMP but for now the presence of drift positions can be used. In Fig. 5 there are 133 stations reporting 300 or more levels: 13% of total radiosonde stations or 16% of those that report temperature. It can be seen that many of the high-resolution native BUFR reports currently come from Europe. (The United Kingdom was producing reports in an alternative BUFR template that does not include pressure but is in the process of changing to the widely used template that does include pressure.) Mexico and Canada provide native BUFR reports with about 150 and 200 levels, respectively (slightly higher than the corresponding TEMP resolution). Canada has slow communication links from some of its remote Arctic stations but intends to improve the resolution from other stations, as well as to provide native BUFR reports from those Canadian stations not yet producing BUFR. Russia is providing native BUFR reports from a few stations. These stations report up to about 1000 temperature levels but only about 150 wind levels. The latter is due to a compromise between accuracy and resolution for the radar wind-finding systems used. Work on various technical aspects is proceeding before BUFR reporting commences at the other Russian stations—over 100 of them. Australia started sending high-resolution native BUFR in two phases: July and October 2015. Some high-resolution reports from the United States are expected on the GTS in 2017. Both Japan and China have started work on producing high-resolution native BUFR but it will take several years before full operational implementation. Previously, Japan’s strategy had been based on reformatted TEMP and the same is probably true of various other countries. The plans and time scales of many countries are still unknown. Various countries upgraded to the new Vaisala RS41 radiosonde in 2014/15; it is disappointing to note that some of them continue to send reformatted TEMP reports. In early 2015, about 70% of native BUFR reports were from Vaisala radiosondes, with smaller proportions from MétéoModem (Ury, France), Graw (Nuremburg, Germany), InterMet (Grand Rapids, Michigan and Cape Town, South Africa), and Russian manufacturers. A few stations provide high-resolution BUFR reports from MétéoModem software. France and some other countries use Météo-France software to produce BUFR reports (at low resolution but including drift positions) from the MétéoModem 1-s raw data. With minor exceptions the native BUFR reports are of good quality. The main problem is that currently they only come from about 20% of radiosonde stations.
Summary of BUFR radiosonde reports for 1–31 Dec 2015 decoded at the Met Office, plotted by station. The color indicates the maximum number of levels per report (gray, 1–29; light blue, 30–99; dark blue, 100–299; green, 300–999; orange, 1000–2999; and purple, 3000 or more; the percentages in the key are relative to the number of stations plotted). Symbols are as in Fig. 3.
Citation: Bulletin of the American Meteorological Society 97, 11; 10.1175/BAMS-D-15-00169.1
As in Fig. 4, but restricted to stations with valid drift positions (native BUFR).
Citation: Bulletin of the American Meteorological Society 97, 11; 10.1175/BAMS-D-15-00169.1
Thus, at present most BUFR radiosonde reports are reformatted TEMP reports. Unfortunately, most of these are not readily usable by NWP centers because of quality problems and/or because they are split into parts. Reformatting errors are common, partly because of the complexity of TEMP code, and have been documented by Ingleby and Vasiljevic (2015) and WMO (2015b). Reformatted PILOT reports perpetuate the regrettable practice whereby some winds are on (nominal) pressure levels and others are on height levels. In BUFR wind-only reports should use one vertical coordinate throughout. A few countries are producing merged, well-reformatted reports, but in general they only provide the complete ascent, without the required preliminary report up to 100 hPa, and occasionally with missing sections. Concern has been raised that some countries do not have the resources to do anything other than reformat TEMP messages. They may need outside help; for example, GCOS and the U.K. voluntary cooperation program enabled the station at Gan (Maldives) to be repaired and configured to send high-resolution BUFR as well as TEMP messages (T. Oakley 2015, personal communication).
RADIOSONDE PROCESSING BEFORE TRANSMISSION.
Radiosonde measurements are processed in various ways before transmission on the GTS (see WMO 2014, chapters I.12 and I.13). The WMO radiosonde intercomparison reports (most recently Nash et al. 2011, hereafter N2011) and Dirksen et al. (2014) provide further details and also an indication of recent trends in the instruments and processing. One trend has been the use of GPS replacing older wind-finding technologies [e.g., radars and radio theodolites; Brettle and Galvin (2003)]. Because the radiosonde swings like a pendulum below the balloon, the raw data are filtered to remove this pendulum motion as far as possible. During daytime there can be occasional temperature spikes as the radiosonde passes through the wake of the balloon, which are also removed by filtering. Most manufacturers apply radiation bias corrections to the temperature, largely to account for daytime overheating in the upper troposphere and stratosphere (N2011, 62–65 and 76–78). The bias corrections usually use climatological average conditions (including cloudiness) and thus may overcorrect during some conditions and undercorrect during others. More recently, based on work by Miloshevich et al. (2004, 2009) and Vömel et al. (2007), Vaisala has introduced humidity corrections both for radiation biases and time-lag effects (due to slow sensor response at very low temperatures). An extra temperature sensor built into the humidity sensor can be used to correct humidity radiation biases at the source; this is used in several radiosonde types including the new Vaisala RS41. The best operational radiosondes now produce usable humidity measurements in the tropical upper troposphere (N2011). Some other sensors react so slowly below about −40°C that their humidities are unusable at such temperatures (but they should still be reported). There can also be problems with ice forming on the humidity sensor during passage through clouds (N2011, p. 114). Ingleby and Edwards (2015; see also the online supplemental material to that article) describe a check for such contamination although in practice a cutoff at −40°C for susceptible radiosonde types removes most of the worst-affected humidities.
Another recent trend is that some GPS radiosondes do not carry a pressure sensor. Instead, pressure is calculated via the hydrostatic equation from surface pressure and profiles of height, temperature, and humidity; this gives marginally worse pressures at low levels but improved pressures at upper levels (N2011). This is not a new idea; Soviet/Russian radiosondes have not used pressure sensors for decades (Zaitseva 1993), but at low radar elevation angles the heights and hence the pressures have larger uncertainty (Kats et al. 2005); this is not a problem for GPS radiosondes.
Especially for climate studies and reanalyses, it is useful to know as much as possible about the radiosonde details and prior processing. Both TEMP and BUFR reports provide the “radiosonde type,” specifying the manufacturer and radiosonde model, such as Vaisala RS92. However, the RS92 underwent changes to the humidity sensor mounting and coating that affected its upper-tropospheric humidity biases (Wang et al. 2013) and also the December 2010 processing changes, implemented much later in some countries (2015 in Germany and the United Kingdom). Yu et al. (2015) also discuss humidity corrections and the importance of the associated metadata. To track such changes, one needs to know the radiosonde serial number and the version of the processing software applied; for now these are appended to Vaisala BUFR reports, but because they are not part of the main template, they are not currently archived at ECMWF and other centers. One also needs to access the Vaisala sounding continuity website to interpret the serial number and processing software version. In general, manufacturer’s algorithms are proprietary and are not made available to users. GRUAN (Dirksen et al. 2014) is producing open-source algorithms to enable a small subset of high quality radiosondes to be used as reference measurements.
USE IN NWP.
In the early years of NWP the analysis was generally performed on standard levels, relying heavily on standard-level radiosonde height and wind data. In the 1980s and early 1990s, ECMWF, the Met Office, and others changed to perform analyses on model levels along with a move to use radiosonde temperatures, including significant-level values, instead of heights. ECMWF and most other centers used significant-level values directly whereas the Met Office interpolated between significant levels to create a complete profile (see the “Significant versus Other Levels” sidebar; direct use of the significant levels is suboptimal, as interpolation between significant levels is more complex but makes better use of the data). In general, NWP centers merge the different TEMP parts into a complete profile to aid in processing and quality control; having all the data in a single BUFR report is a significant advantage. ECMWF, the Met Office, and FNMOC/NRL use the radiosonde type to decide whether to use upper-tropospheric humidity. It can also be used in setting observation uncertainties and bias correction [see Agustí-Panareda et al. (2009); however, Milan and Haimberger (2015) suggest that radiosonde type may be less useful for bias correction than previously thought]. Because modern radiosonde systems generally have radiation corrections applied before transmission, the need for NWP to bias correct current data is diminishing, but reanalysis systems have to bias correct older radiosonde types. There is a trend toward earlier NWP data cutoff times, and at the Met Office there have been discussions about the production of a report when the radiosonde reaches 500 hPa, or perhaps triggered shortly before a cutoff time. Particularly if 1-h analysis windows are used for limited-area NWP, then a radiosonde ascent can span two or more analysis windows. This raises various practical issues.
SIGNIFICANT VERSUS OTHER LEVELS
Many NWP systems do not interpolate between significant levels as they were intended to be used (Table 1). However, using significant levels directly (effectively as single-level data) is suboptimal and is somewhat analogous to treating the daily minimum–maximum temperatures as if they were the mean nighttime–daytime temperatures at a surface station. Also note that there can be large gaps (where the profiles are close to linear) between significant levels; interpolation is needed to fill in such gaps. Figure SB1 shows observation minus background (short-range forecast) statistics for temperature. As expected, the statistics are very similar between the standard and high-resolution levels; however, they are markedly larger/worse for significant levels. The main reason for this is not the instrument errors but the representativeness errors: the difference between point values and values averaged over the volume represented by the model variable. With its finite resolution the model cannot fully represent the tropopause temperature, the maximum wind, or the other local extrema that are included as significant levels. Thus, direct use of the tropopause significant level may result in a local bias (observation cooler than background). For wind and humidity (not shown), the observation minus background statistics are also worse for significant levels, but the difference is particularly large for stratospheric temperatures, possibly because of vertical temperature oscillations in stable conditions. This issue concerning the use of significant levels is less well known than it should be and, among other things, suggests that when thinning BUFR reports for assimilation it is better to select points at random than to choose significant levels. Also, because significant levels imply the profile between the significant levels [only the Met Office–BoM system takes account of this, as documented by Ingleby and Edwards (2015) but implemented much earlier], a simple comparison of the numbers of levels between TEMP and high-resolution reports is misleading.
The rms of observation minus ECMWF background temperature (K) for German stations, July 2015. Background values are interpolated to the reported levels; then, statistics are calculated for layers around standard levels (with the addition of a 5-hPa level): blue, TEMP significant levels; black, TEMP standard levels; and red, BUFR levels (some preliminary thinning has been applied to the 2-s profile but there are still thousands of levels per ascent). There are 14 German stations using Vaisala RS92 radiosondes, providing about 1100 ascents per month in total.
Citation: Bulletin of the American Meteorological Society 97, 11; 10.1175/BAMS-D-15-00169.1
In strong wind conditions radiosondes can drift 200 km or more; Fig. 6 shows an example. Treatment of radiosonde drift (Laroche and Sarrazin 2013; Ingleby and Edwards 2015; Choi et al. 2015) shows promising results. It is possible to estimate the drift from the reported winds and an assumed ascent rate, but it is much more satisfactory to use reported drift from BUFR. In rapidly developing situations, knowing the precise time and location of observations is essential for NWP and the approximations in TEMP reporting are outdated.
Radiosonde drift from Yarmouth, NS, Canada on two consecutive days; the color scale shows pressure (hPa). White squares mark other radiosonde stations. At 44°N, 3° of longitude cover about 240 km. (Courtesy of D. Tyndall, NRL.)
Citation: Bulletin of the American Meteorological Society 97, 11; 10.1175/BAMS-D-15-00169.1
Launch-position errors in the reports can degrade analyses and forecasts. To protect against this, ECMWF, JMA (for land surface reports), and FNMOC/NRL currently replace some or all BUFR positions with those from their position lists, based largely on WMO Publication 9A. (Over time, the reliability of the BUFR positions should improve and the need for this step is expected to diminish.)
The Met Office NWP system was extended to process high-resolution BUFR data by Ingleby and Edwards (2015). ECMWF and some other centers thin high-resolution profiles before assimilating them, trying to keep an approximately constant number of data per model layer. Both centers “activate” reports by station identifier. If the BUFR report is active, then a duplicate check deactivates any matching TEMP reports (the check has to allow for time differences: TEMP often has the nominal launch time, such as 1200 UTC, whereas BUFR has the actual launch time, which can be 1115 UTC or earlier). The list of active stations/formats is controlled by a file that is generally updated monthly. Any more complex change (such as processing a new format or modifying duplicate/quality checks) involves changing the source code of the forecasting system; except in an emergency, this is only done once or twice a year as part of an extensively tested package involving changes to many other components of the forecasting system. NWP centers do not want to lose data (if TEMP reports are switched off before they are ready to use the BUFR replacements), but they are also concerned with avoiding acute problems, for example, failures of the forecasting system due to incorrect/unexpected input data. Such failures can be caused by the sheer number of levels in a few reports (up to 14,500 levels in one report) or coding errors introduced by data producers.
Recently, FNMOC/NRL started using most BUFR radiosonde reports (including those split into parts that most other NWP centers consider problematic). Météo-France uses all native BUFR reports, essentially those in Fig. 5, accounting for about 22% of the global TEMP stations. ECMWF and several other NWP centers use BUFR reports from the high-resolution subset and two centers are not using BUFR radiosonde reports yet (see Table 1). In the short term there are three basic approaches: 1) use as much BUFR as possible, 2) use as little BUFR as possible (ascents only available in BUFR), and 3) introduce extra BUFR subsets gradually after careful checking. All the centers have some room for improvement in their use of radiosonde data. The use of BUFR data has to extend to other systems as well: graphical display, verification, observation monitoring, and reanalysis (also surface and oceanographic analysis systems). A few radiosonde reports are prepared in CREX, an equivalent of BUFR in character form. To the best of our knowledge, no NWP centers are using the interim CREX format.
Radiosonde data in use at various global NWP systems. Shown is information on use of significant levels, where direct use indicates that it is treated in the same way as a single-level report and interpolated/averaged indicates interpolation between radiosonde reported levels and averaging over model layers (see Ingleby and Edwards 2015); date that use of BUFR radiosonde data started/will start; the approximate percentage of radiosonde stations for which BUFR data were actively used in early 2016; the treatment of radiosonde drift; and additional notes. The global models currently have between about 50 and 150 vertical levels but this number increases over time. Note that the Météo-France system assimilates data in 30-min time slots. BUFR with drift positions is assimilated in 30-min segments, with each segment being treated as vertical, but using the horizontal position of the lowest point of the segment.
Starting in early 2014, various NWP centers have been providing feedback to data producers on issues with their BUFR data and some information is being exchanged online (https://software.ecmwf.int/wiki/display/TCBUF/). Because of differences in GTS routing and BUFR decoders, centers can see different subsets of the data and one center may decode a report that another has trouble with. Some data producers have been correcting errors and users have been improving decoders, but even “simple” changes can take months. For NWP centers it is important to have about two months’ worth of notice (via METNOs or in future WIS) of the cessation of alphanumeric reports; this allows time for the quality of the BUFR reports to be checked and for them to be activated, as well as time to provide feedback on problems to the data producer. The feedback can be sent either directly or via WMO (or via EUMETNET for European reports) and can relate to either coding or availability. For one country, the 0000 UTC BUFR reports were available on the GTS but the 1200 UTC reports were not; comparison with the alphanumeric reports is very helpful in identifying such issues. In the longer term the monitoring of BUFR reports has to become part of the regular quality control process and availability monitoring that was introduced by Hollingsworth et al. (1986; also, see the supplemental material for links to monitoring information).
Over time, radiosonde data have become relatively less important as the volume and usage of satellite data has improved dramatically; aircraft and wind profiler data also contribute in some areas. However, radiosonde data are still very important, both because of their direct impact and because they act as a reference for satellite and aircraft data (Radnoti et al. 2012). To quote from Andersson and Sato (2012),
Several presentations focused on the importance of in situ profiling observations for regional NWP. Significant benefit is obtained from radiosonde ascents together with wind and temperature profiles from ascending or descending aircraft…The impact of radiosonde data is expected to increase further once the reporting of the complete time and position information as well as high vertical resolution have been widely adopted and utilized in assimilation.
CONCLUSIONS.
Some high-resolution BUFR reports are currently available, primarily from European and Australian stations. The United States, Russia, Japan, and Canada are all working on the provision of high-resolution reports. For most other countries it is unclear if/when high-resolution reports will be available and in some cases software development by the radiosonde manufacturer is needed first. The alphanumeric TEMP reports will eventually cease to be sent and used. Some, from ships, stopped in November 2014, so NWP centers and other users need to start using the BUFR reports. When generated directly from the original data, BUFR reports have advantages of extra precision, better metadata, and balloon drift position information. Treatment of balloon drift shows some improvement to NWP. Most centers can also benefit from using (randomly sampled) high-resolution profiles in place of assimilating significant levels, as if they were single-level observations. For short-range forecast verification, extra height precision in BUFR is helpful at some levels. However, most of the BUFR reports currently available have been generated by reformatting TEMP messages. Unfortunately, many of these do not meet the BUFR coding regulations (see the “Valid and Invalid BUFR“ sidebar) and/or contain conversion errors and are therefore unusable at most major NWP centers.
NWP centers only started extensively checking and using the BUFR data in 2014. They have provided feedback to the data producers, resulting in improvements to BUFR availability and position metadata, although some data producers have yet to take action. It is important for NWP centers that they receive several months’ notice of the withdrawal of alphanumeric reports; they need to know that the BUFR data from a particular country are stable, timely, and of good quality before using them operationally. It is highly desirable for data producers to have their BUFR reports (availability, metadata, contents, and conformance with BUFR regulations) reviewed by other NMSs or NWP centers well before the corresponding alphanumeric reports are terminated. Timeliness of data receipt is also important, as regional NWP systems have very tight data cutoff times. Better communication by WMO (perhaps including a regular 6-monthly summary of recent and forthcoming changes to the World Weather Watch) would be helpful.
There are some tensions between data producers (especially those who have based their strategy on reformatting TEMP reports) and data users, as well as between the desire to “complete” the migration as soon as possible and the aspiration that all reports should contain good-quality high-resolution reports. The migration is complex because almost 200 countries are involved and within each NMS changes by several teams are required (advice from larger NMSs is generally available). Radiosonde data are still very important for NWP and other uses and there is still much work to do to ensure a successful migration and to make good use of the extra information available in BUFR. We would encourage data producers to work toward high-resolution BUFR data as soon as possible and to notify users in good time of such changes.
ACKNOWLEDGMENTS
Various colleagues at ECMWF and the Met Office, plus three reviewers, have provided useful comments on the manuscript. Many more people are involved in the migration to BUFR.
APPENDIX. Key acronyms used in this article.
| ASAP | Automatic Shipboard Aerological Programme |
| BoM | Bureau of Meteorology |
| BUFR | Binary Universal Form for the Representation of Meteorological Data (a TDCF) |
| CBS | Commission for Basic Systems (of WMO), parent body of IPET-DRMM |
| CMC | Canadian Meteorological Centre |
| CREX | A character TDCF |
| DWD | Deutscher Wetterdienst |
| ECMWF | European Centre for Medium-Range Weather Forecasts |
| EUMETNET | Grouping of 31 European NMSs that cooperate on some activities including observing systems |
| FNMOC/NRL | Fleet Numerical Meteorology and Oceanography Center (operations)/Naval Research Laboratory (development) |
| GCOS | Global Climate Observing System |
| GPS | Global positioning system |
| GRUAN | GCOS Reference Upper Air Network |
| GTS | Global Telecommunications System |
| JPET-DRMM | The Inter-Programme Expert Team on Data Representation Maintenance and Monitoring |
| JMA | Japan Meteorological Agency |
| METNO | Meteorological notifications of changes to information transmitted on the GTS |
| NCEP | National Centers for Environmental Prediction |
| NMS | National (hydro)meteorological service |
| NWP | Numerical weather prediction |
| PILOT | Alphanumeric code for wind-only reports |
| SYNOP | Surface synoptic observation |
| TAC | Traditional alphanumeric codes (including TEMP and PILOT) |
| TDCF | Table-driven code form |
| TEMP | Alphanumeric code for radiosonde reports of temperature, humidity, wind, and height |
| WIGOS | WMO Integrated Global Observing System |
| WIS | WMO Information System |
| WMO | World Meteorological Organization |
REFERENCES
Agustí-Panareda, A., and Coauthors, 2009: Radiosonde humidity bias correction over the West African region for the special AMMA reanalysis at ECMWF. Quart. J. Roy. Meteor. Soc., 135, 595–617, doi:10.1002/qj.396.
Andersson, E., and Y. Sato, Eds., 2012: Final report of the Fifth WMO Workshop on the Impact of Various Observing Systems on Numerical Weather Prediction. WMO Integrated Global Observing System, 6 pp. + appendixes. [Available online at www.wmo.int/pages/prog/www/OSY/Meetings/NWP5_Sedona2012/Final_Report.pdf.]
Brettle, M. J., and J. F. P. Galvin, 2003: Back to basics: Radiosondes: Part 1–The instrument. Weather, 58, 336–341, doi:10.1256/wea.126.02A.
Choi, Y., J.-C. Ha, and G.-H. Lim, 2015: Investigation of the effects of considering balloon drift information on radiosonde data assimilation using the four-dimensional variational method. Wea. Forecasting, 30, 809–826, doi:10.1175/WAF-D-14-00161.1.
Ciesielski, P. E., and Coauthors, 2014: Quality-controlled upper-air sounding dataset for DYNAMO/CINDY/AMIE: Development and corrections. J. Atmos. Oceanic Technol., 31, 741–764, doi:10.1175/JTECH-D-13-00165.1.
Dirksen, R. J., M. Sommer, F. J. Immler, D. F. Hurst, R. Kivi, and H. Vömel, 2014: Reference quality upper-air measurements: GRUAN data processing for the Vaisala RS92 radiosonde. Atmos. Meas. Tech., 7, 4463–4490, doi:10.5194/amt-7-4463-2014.
Hamilton, K., and R. A. Vincent, 1995: High-resolution radiosonde data offer new prospects for research. Eos, Amer. Geophys. Union, 76, 497–506, doi:10.1029/95EO00308.
Hollingsworth, A., D. B. Shaw, P. Lönnberg, L. Illari, K. Arpe, and A. J. Simmons, 1986: Monitoring of observation and analysis quality by a data assimilation system. Mon. Wea. Rev., 114, 861–879, doi:10.1175/1520-0493(1986)114<0861:MOOAAQ>2.0.CO;2.
Ingleby, B., and D. Edwards, 2015: Changes to radiosonde reports and their processing for numerical weather prediction. Atmos. Sci. Lett., 16, 44–49, doi:10.1002/asl2.518.
Ingleby, B., and D. Vasiljevic, 2015: Progress report on migration to BUFR. ECWMF. [Available online at https://software.ecmwf.int/wiki/display/TCBUF/ECMWF.]
Kats, A., A. Balagourov, and V. Grinchenko, 2005: The impact of new RF95 radiosonde, introduction on upper-air data quality in the north-west region of Russia. Tech. Conf. on Meteorological and Environmental Instruments and Methods of Observation, Bucharest, Romania, WMO. [Available online at www.wmo.int/pages/prog/www/IMOP/publications/IOM-82-TECO_2005/Posters/P2%2807%29_Russia_Kats.pdf.]
Laroche, S., and R. Sarrazin, 2013: Impact of radiosonde balloon drift on numerical weather prediction and verification. Wea. Forecasting, 28, 772–782, doi:10.1175/WAF-D-12-00114.1.
Milan, M., and L. Haimberger, 2015: Predictors and grouping for bias correction of radiosonde temperature observations. J. Geophys. Res. Atmos., 120, 10 736–10 766, doi:10.1002/2015JD023635.
Miloshevich, L. M., A. Paukkunen, H. Vömel, and S. J. Oltmans, 2004: Development and validation of a time-lag correction for Vaisala radiosonde humidity measurements. J. Atmos. Oceanic Technol., 21, 1305–1327, doi:10.1175/1520-0426(2004)021<1305:DAVOAT>2.0.CO;2.
Miloshevich, L. M., H. Vömel, D. N. Whiteman, and T. Leblanc, 2009: Accuracy assessment and correction of Vaisala RS92 radiosonde water vapor measurements. J. Geophys. Res., 114, D11305, doi:10.1029/2008JD011565.
Nash, J., T. Oakley, H. Vömel, and L. I. Wei, 2011: WMO intercomparison of high quality radiosonde systems (Yangjiang, China 12 June–3 August 2010). WMO Instruments and Observing Methods Rep. 107, 238 pp. [Available online at www.wmo.int/pages/prog/www/IMOP/publications-IOM-series.html.]
Radnoti, G., P. Bauer, A. McNally, and A. Horanyi, 2012: ECMWF study to quantify the interaction between terrestrial and space-based observing systems on numerical weather prediction skill. ECMWF Tech. Memo. 679, 97 pp. [Available online at www.ecmwf.int/en/elibrary/11814-ecmwf-study-quantify-interaction-between-terrestrial-and-space-based-observing.]
U.S. Weather Bureau, 1949: Radiosonde and rawinsonde code, 1949 edition (in international form as adopted by the International Meteorological Organization at Paris in 1946 and revised at Washington in 1947). USWB.
Vömel, H., and Coauthors, 2007: Radiation dry bias of the Vaisala RS92 humidity sensor. J. Atmos. Oceanic Technol., 24, 953–963, doi:10.1175/JTECH2019.1.
Wang, J., L. Zhang, A. Dai, F. Immler, M. Sommer, and H. Vömel, 2013: Radiation dry bias correction of Vaisala RS92 humidity data and its impacts on historical radiosonde data. J. Atmos. Oceanic Technol., 30, 197–214, doi:10.1175/JTECH-D-12-00113.1.
WMO, 2014: Guide to meteorological instruments and methods of observation (WMO Rep. 8). [Available online at www.wmo.int/pages/prog/www/IMOP/IMOP-home.html.]
WMO, 2015a: Manual on codes: Volumes I.1 and Volume I.2. WMO/World Weather Watch. [Available online at www.wmo.int/pages/prog/www/WMOCodes.html.]
WMO, 2015b: Status report on migration to table-driven code forms. Inter-Programme Expert Team on Data Representation Maintenance and Monitoring Rep., 14 pp. [Available online at www.wmo.int/pages/prog/www/WMOCodes/MigrationTDCF/PR6868/2015-09-25-PR-6868-OBS-WIS-DRMM-MIGRATION_en.pdf.]
Yu, H., P. E. Ciesielski, J. Wang, H.-C. Kuo, H. Vömel, and R. Dirksen, 2015: Evaluation of humidity correction methods for Vaisala RS92 tropical sounding data. J. Atmos. Oceanic Technol., 32, 397–411, doi:10.1175/JTECH-D-14-00166.1.
Zaitseva, N. A., 1993: Historical developments in radiosonde systems in the former Soviet Union. Bull. Amer. Meteor. Soc., 74, 1893–1900, doi:10.1175/1520-0477(1993)074<1893:HDIRSI>2.0.CO;2.
