1. Introduction
Recent years have seen an increasing demand for high- (1–5 km) resolution meteorological analyses at the National Oceanic and Atmospheric Administration’s (NOAA) National Weather Service (NWS) and in the environmental community at large. In particular, NWS forecasters use high-resolution analyses of sensible weather elements to help create and verify the gridded forecasts of its National Digital Forecast Database (NDFD; Glahn and Ruth 2003). High-resolution analyses are also increasingly necessary for mesoscale modeling and forecasting. They are also sought for climate-related applications, including climate change impact studies on sectors of critical importance to society such as human health, agriculture, energy, water resources, and coastal systems. Furthermore, the finescale details contained in these fields enhance the representativeness of the physical drivers in air quality models, forest fire models, and dispersion models used, for example, to simulate the transport of pollutants and other hazardous materials. The creation of high-resolution analyses is also in tune with the current efforts to improve, expand, and strengthen the capabilities of the nation’s mesoscale observational networks (National Research Council 2009).
To help meet its demand for high-resolution analyses, the NWS in 2004 established a multiphased program to build the Reanalysis of Record (Horel and Colman 2005). The Reanalysis of Record is an approximate 30-yr record of high spatial and temporal resolution three-dimensional reanalyses paralleling the global National Centers for Environmental Prediction–National Center for Atmospheric Research 40-Year Reanalysis (NCEP–NCAR; Kalnay et al. 1996) and the North American Regional Reanalysis (Mesinger et al. 2006).
The Real-Time Mesoscale Analysis, or RTMA, is the program’s initial phase, intended as a proof of concept and baseline for subsequent developments toward the Reanalysis of Record. The RTMA capitalizes on and enhances the existing analysis capabilities at NWS to produce 6–1.25-km-resolution analyses of near-surface conditions on grids that match those of the NDFD. Among other applications, the RTMA is used at various Weather Forecast Offices (WFOs) as a tool to help initialize and adjust their digital forecasts within the framework of the Interactive Forecast Preparation System (IFPS; Ruth 2002) on the Advanced Weather Interactive Processing System (AWIPS; Seguin 2002). It also provides a means by which to assess the quality of the NDFD forecasts, as well as a reference data source to correct biases in model forecasts (Z. Toth 2007, personal communication).
This work reports on the development of the RTMA at NCEP, Earth System Research Laboratory (ESRL), and National Environmental, Satellite, and Data Information Service (NESDIS). The RTMA was implemented at NCEP Central Operations in 2006 for a domain matching the conterminous U.S. NDFD 5-km grid (CONUS-RTMA) and in 2008 for domains matching the 6-km Alaska, 2.5-km Hawaii, and 2.5-km Puerto Rico NDFD grids (Fig. 1). Preparations are under way to run the system for the Guam NDFD grid as well. In all cases, the system produces two-dimensional variational (2DVAR) analyses of 2-m temperature, 2-m specific humidity, 2-m dewpoint, surface pressure, and 10-m U and V wind components, and computes an estimate of the analysis uncertainty for each analyzed variable. In addition, the CONUS-RTMA maps the NCEP stage II precipitation amount and the NESDIS effective cloud amount from Geostationary Operational Environmental Satellite (GOES) sounders onto its 5-km grid. The analysis is performed hourly in all applications, the valid analysis time being the top of the hour. The fields are available at around 50 min past the hour and disseminated to NWS forecasters and external users through AWIPS and the National Digital Guidance Database, a sister to the NDFD.
The four domains in which the RTMA currently runs, namely, CONUS, Alaska, Puerto Rico (PR), and Hawaii (HI) NDFD grids. For illustrative purposes only, the figure shows the temperature (°C) valid at 1800 UTC 19 May 2010. For the CONUS domain, the projection on the plane is of the Lambert conformal conic type with the following parameters: the orientation longitude is 265°E; the first and second secant latitudes, as well as the latitude where the grid spacing is defined, are 25°N; the grid spacing in both the x and y directions is 5079.406 m; the latitude and longitude of the first grid point are 20.19199°N and 238.4459°E, respectively; and the number of grid points along the x and y directions are 1073 and 689, respectively. This grid coincides with the CONUS NDFD grid, and is known as AWIPS Grid 197 in NCEP’s Grid Identification Table (information online at http://www.nco.ncep.noaa.gov/pmb/docs/on388).
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
This paper focuses on the CONUS-RTMA. It describes its 2DVAR component in detail and provides a brief description of the derivation of the 5-km resolution precipitation and sky-cover fields. In the remainder of this work, the term “RTMA” implicitly denotes the CONUS version. The paper is organized as follows. Section 2 introduces the 2DVAR analysis. Section 3 describes the Rapid Update Cycle (RUC)-derived background fields, the observations, and their quality control. The Lanczos-based estimate of the 2DVAR analysis uncertainty is described in section 4. Section 5 presents three case studies that illustrate the performance of the 2DVAR and presents statistics of the fit of the analysis to the observations evaluated with the help of the cross-validation technique. Section 6 describes the mapping of the stage II precipitation amount and NESDIS GOES sounder effective cloud amount onto the RTMA domain. Section 7 describes the system improvements that are scheduled to be implemented during the next RTMA upgrade cycle, as well as the current developmental work. The conclusions follow in section 8.
2. The RTMA-2DVAR
a. The analysis system and the use of recursive filters
The background error covariance matrix 
The spatial correlation scales and variances of background errors are latitude dependent and represent a rescaling of those used with the NCEP Nonhydrostatic Mesoscale Model (NMM; Janjić et al. 2001; Janjić 2003) data assimilation system. The rescaling is performed in an extended parameter space that includes the Lf parameters, which for simplicity are assumed constant over the analysis domain. Their initial values are chosen empirically. The rescaling attempts to lower the root-mean-square error of the fit of the analysis to the observations while avoiding overfitting the observations or producing fields of unrealistically small-scale analysis increments. The feedback provided by the WFOs on the RTMA performance plays an important role in the hitherto trial and error rescaling. The domain-averaged values of the spatial correlation scales, background error variances, and function correlation scales are listed in Table 1.
The horizontal spatial correlation scale Lh, function correlation scale Lf, and background error standard deviation σb used for streamfunction ψ, velocity potential χ, temperature T, pseudo–relative humidity (RH)pseudo, and surface pressure Psfc.
The use of a terrain-following autocovariance would seem intuitively appealing for representing background errors in temperature, surface pressure, and moisture fields, since these fields themselves tend to exhibit a strong dependency on the terrain. The procedure, however, becomes more difficult to justify for errors in the background wind, whose circulations often include flows going up and down mountain slopes. For that reason, the RTMA chooses the Lf parameters such that the anisotropy is very weak for the wind analysis.
Figure 2 shows the autocorrelation functions for temperature and streamfunction for a selected “anchor point” located in a region of strong terrain gradient. The selected Lf parameters lead to a very pronounced anisotropy for temperature and a nearly isotropic autocorrelation for streamfunction. Results for surface pressure and pseudo–relative humidity are qualitatively similar to those for temperature and are therefore not shown. Similarly, the results for velocity potential closely resemble those for streamfunction and are therefore also not displayed.
Autocorrelation functions for (a) temperature and (b) streamfunction for the anchor point marked by the dot. From the outermost to the innermost contours, the autocorrelation values are 0.1, 0.2, 0.5, 0.7, and 0.8. Shaded contours show the underlying terrain used to construct the autocorrelations. Values on the x and y axes are in grid units, whereby (x = 1, y = 1) would correspond to the southwestern-most grid point of the RTMA domain. It is noted that the partial domain of (b) is larger than that of (a).
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
b. The handling of land–water contrasts
Analyzing or retaining the correct land–water contrasts can pose a challenge. This is an important aspect especially when analyzing temperature in the vicinity of shorelines. For such cases, an algorithm is desirable to restrict the influence of the warm (cold) land observations on the nearby cold (warm) water. Similarly, marine observations should have their influence mainly restricted to the water body. In the 2DVAR scenarios, this is achieved by sharpening the background error autocovariances along shorelines. The algorithm decreases the effective elevation height on major water bodies arbitrarily by 500 m in the autocovariance construction to sharpen the terrain gradient along the shorelines. Figure 3 illustrates the effects of this artifact on the analysis of a single temperature observation located on the eastern shore of Lake Michigan. Figure 3a shows the analysis increments in the absence of the artificial terrain gradient sharpening, and as expected, the contour lines cross the shoreline into the lake. Figure 3b shows the effects of sharpening the terrain gradient along the shoreline. As desired, the analysis increments are almost exclusively confined to the landmass and display a large gradient near the shoreline as the amplitudes quickly approach zero going into the lake.
Temperature analysis increments (°C) from a single Mesonet temperature observation located on the eastern shore of Lake Michigan. (a) Case in which no artificial sharpening of the terrain gradient is performed along the shoreline and (b) case in which the sharpening is used. The dot represents the location of the observation. The analysis time for this illustration is of no consequence.
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
3. The RTMA background, the observations, and their quality control
a. The RTMA background
The CONUS-RTMA 2DVAR uses 1-h forecasts from the 13-km Rapid Update Cycle (RUC; Benjamin et al. 2004), downscaled to the 5-km grid as its background. The RUC is a hybrid sigma-isentropic coordinate 3DVAR assimilation system developed at ESRL. Its detailed hourly assimilation of a large suite of both conventional and nonconventional observations, as well as the use of a digital-filter initialization that guarantees a 1-h forecast that is nearly free from model spindown, makes the RUC particularly appropriate for providing the background for further GSI-2DVAR enhancement. The downscaling procedure uses the 3D atmospheric–land surface representation provided by the RUC to generate surface fields that are physically consistent with land surface conditions, land–water contrasts, and terrain elevation of the RTMA grid. The procedure also attempts to preserve other 3D effects that are present in the RUC, such as realistic thermal stability, boundary layer structure, and local circulations. The downscaled RUC fields that are currently used in the RTMA are 2-m temperature, 2-m dewpoint, 2-m specific humidity, 10-m U and V wind components, and surface pressure. The downscaling is detailed in Benjamin et al. (2007). It comprises a bilinear horizontal interpolation of the RUC fields to the higher-resolution RTMA grid, the use of the near-surface vertical gradients from the RUC to adjust the interpolated fields to the RTMA 5-km terrain, and a sharpening of the expected coastline gradients in a manner consistent with the land–water-type indicators from the 5-km RTMA grid. The 5-km terrain of the RTMA, as well as the vegetation type from which the land–water-type indicators are derived, are extracted from the Weather Research and Forecasting Model (WRF) standard initialization program, without any smoothing applied.
For a selected October 2008 day, Fig. 4 illustrates the effects of the downscaling on the temperature field over a portion of the RTMA domain that includes the Rocky Mountains. Not surprisingly, the downscaling introduces local small-scale features reflecting the higher-resolution terrain (not shown). The effects of the terrain adjustment are also apparent in the wind speed displayed in Fig. 5. Compared with the RUC wind speed, the RTMA background displays small-scale patches of enhanced wind speed for most of western Colorado, over the Rocky Mountains.
(a) The original 13-km resolution 1-h RUC temperature forecast and (b) its 5-km downscaled version over a portion of the RTMA domain that includes the Rocky Mountains. Units are °C and the example is for the forecast starting at 1700 UTC 10 Oct 2008.
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
As in Fig. 4, but for the 10-m wind speed displayed on a smaller domain that focuses mainly on the state of CO. Units are m s−1.
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
As local flow features are enhanced, it is also important to ensure that at the minimum the downscaling algorithm does not degrade the domain-averaged statistics of the fit of the fields to the observations. Ideally, the procedure should in fact improve the statistics. Figure 6 compares how the RTMA first guess and the original RUC field fit the observations. The latter are the full pool of quality-controlled observations used in the 2DVAR assimilation. The statistics are for a period of 30 days, from 0000 UTC 2 July 2009 to 2300 UTC 31 July 2009. They represent time and domain-averaged root-mean-square errors (RMSEs) for each hour of the day. For the period chosen, the downscaling improves the RMSEs for temperature, surface pressure, and wind, and keeps those for the specific humidity nearly unchanged.
The 30-day, domain-averaged RMSE at each hour of the day for the RUC-13 (red lines) and the RTMA background (black lines) for (a) 2-m temperature, (b) 2-m specific humidity, (c) surface pressure, and (d) 10-m wind. The verification is performed against the quality-controlled observations used in the assimilation. It is noted that the verification for surface pressure did not apply any correction for the difference between the observation elevation and the local terrain height.
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
b. Observations and quality control
The observations are grouped into four categories for their assimilation into the GSI, namely, temperature, moisture, surface pressure, and wind. These observations originate from synoptic, aviation routine weather report (METAR), Mesoscale Network (Mesonet), ship, buoy, tide gauge, and Coastal-Marine Automated Network (C-MAN) stations. The feed for Mesonet observations is the Meteorological Assimilation Data Ingest System (MADIS; Miller et al. 2007) and it is the Telecommunication Operations Center for the other observation types. The assimilation time window is −12 to +12 min centered around the analysis time. Its relatively small extent was chosen so that the observations are representative for the valid analysis time. As discussed in section 7, the forthcoming use of the first guess at the appropriate time (FGAT) technique will justify expanding considerably the assimilation time window for all observation types.
Figure 7 displays a representative map of the temperature stations found in the RTMA observation file for a given analysis time. The example shown is from 1500 UTC 20 November 2009. It contains 14 299 stations, of which 12 081 are land Mesonet stations, 2049 are land synoptic and METAR stations, and 169 are surface marine stations (ship, buoy, C-MAN, and tide gauge). Mesonet stations alone account for 84.5% of the stations, while land synoptic and METARs account for 14.3%, and marine stations account for 1.2%. These percentage numbers also hold approximately true for moisture, surface pressure, and wind observations. Likewise, Fig. 7 bears good resemblance with similar maps for these three other observation categories.
A representative map of the temperature stations used at a given analysis time. The example is from the 1500 UTC 20 Nov 2009 analysis. Green dots represent Mesonets, red dots show land synoptic and METAR stations, and blue dots show marine stations. The total number of stations is 14 299.
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
There are four layers of quality control in the RTMA-2DVAR:
Observations preflagged by the MADIS quality control process are also flagged by the RTMA.
The so-called gross-error check is performed at the beginning of each outer loop of the minimization: the absolute value of the ratio of the observation increment to the observation error is evaluated, and the observation is flagged when that ratio exceeds a preestablished threshold. For each observation type, Table 2 summarizes the observation errors and the thresholds used for the gross-error check.
Observation blacklists are used for each observation type. They combine four distinct lists: (i) a static list updated periodically by the WFOs, (ii) a static list composed from the MADIS monthly quality control statistics, (iii) a static list containing stations flagged by the RTMA developers during routine evaluation of the RTMA or following feedback from NWS field forecasters, and (iv) a dynamic list updated hourly by the RTMA. The latter is constructed using the gross-error statistics from the previous three to six analyses, and is applied to the next analysis.
In addition, for Mesonet winds the RTMA uses a list of “trusted providers,” as well as a list of “trusted stations,” both borrowed from the RUC system. Both lists are constructed from long-term statistics of the differences between the observed and the RUC model wind speeds. However, the “provider list” groups the stations according to the names of their providers and subproviders (information online at http://madis.noaa.gov/mesonet_providers.html), while the “station list” treats each station individually (Benjamin et al. 2010). Mesonet winds are only used in the assimilation if they belong to at least one of these lists.
The square root of the observation error covariance (σo) for each observation type found in the input error table for the RTMA. Note that the actual observation error used may be inflated in the RTMA to account for the occurrence of duplicate observations or in response to the input quality control flag from MADIS. Also shown in the table is the ratio R between the maximum acceptable error and σo used in the gross-error check for each observation type. It is also noted that, although specific humidity is the moisture observation, the gross-error check is performed using pseudo–relative humidity (RH). The actual observation error for specific humidity is that of pseudo-RH multiplied by the local value of the background specific humidity.
The percentage of stations flagged by the various quality control mechanisms is around 10% for temperature and moisture, 25% for surface pressure, and a staggering 55% for wind. The latter figure stems mainly from the fact that about 60% of the Mesonet wind stations fail the provider- and station-list criteria for acceptability. The quality control procedure for Mesonet winds is indeed a challenging matter. Compared with the background, these data tend to exhibit a low speed bias arguably due primarily to questionable anemometer siting strategies (Benjamin et al. 2010). However, some of the perceived low speed biases may also be reflective of the lack of enough resolution and adequate boundary layer physics in the background. The example given later in section 5b illustrates the critical importance that quality control plays for the wind analysis.
4. Estimate of the analysis uncertainty
Associated with each analysis is the analysis error covariance matrix, 
Figure 8 displays an example of the analysis error for temperature over a selected region in the states of Utah, Idaho, Wyoming, and Nevada. As expected, the analysis error is equal to the (inflated) background error in data-void regions. At the observation sites and in their vicinities, the analysis error is reduced compared with the background errors. This is the expected effect from the information contained in the observations. It is also apparent that the patterns of the analysis error are locally consistent with the shapes of the underlying, terrain-following background error covariances.
An example of the analysis error for temperature in °C (shaded contours) over parts of UT, ID, and WY. The valid time is 1200 UTC 29 Dec 2009, and the analysis excludes Mesonet observations. The small black triangles represent the observations used in the analysis. Black contour lines represent the topography at 300-m intervals. Actual values of the contour lines are of no consequence.
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
As discussed in appendix B, although the pattern of the analysis error is estimated quite accurately, the magnitude of the error is unfortunately underestimated. Calibration procedures for the error magnitude are still being pursued within the broader context of the GSI. At present, the RTMA analysis uncertainty is intended mainly for identifying regions of “small” and “large” errors in a relative sense.
5. Case studies and validation statistics
a. Analysis of a cold pool in the Washington, D.C., area
The RTMA has been shown to effectively capture convectively induced cold surface temperature pools, which are often absent from mesoscale model analyses and forecasts. Figures 9a and 9b show the RTMA first-guess temperature and analysis, respectively, for 2100 UTC 13 June 2007. The region of interest in this case description is the greater Washington, D.C., area, with emphasis on northeastern Virginia. That region is roughly centered at 38.8°N, 77°W. Figures 9c and 9d display the observations used during the first and second outer loops of the minimization, respectively. The colors of the small circles used to mark the location of the observations follow the color code shown below each panel. In the region of focus, both panels show a cold temperature pool, with many of the observations displaying values between 20° and 24°C in northeastern Virginia. The observations also show a tight temperature gradient, particularly across the District of Columbia and Maryland, with temperatures increasing rapidly away from the cold pool. Comparing Fig. 9a with Figs. 9c and 9d, it becomes apparent that the first guess is too warm in the region of interest, with temperatures ranging from 27° to 29°C. Figure 9b, on its turn, shows the analysis displaying cooler temperatures that correspond very well with the observed values. The cold pool is captured well.
(a) The background temperature and (b) analysis in °C at 2100 UTC 13 Jun 2007. The small circles show the observations used during the (c) first and (d) second outer loops of the minimization. Their colors follow the color code shown at the bottom of the figure. (e),(f) Observations rejected during the first and second outer loops of the minimization, respectively.
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
This example also illustrates the benefit of the two-outer-loop configuration described in section 1 for handling the observation quality control. It is noted that the cold pool is better defined in Fig. 9d than it is in Fig. 9c. The explanation goes as follows. The gross-error check at the start of the first outer loop rejected many of the observations, since they were too cold compared with the first guess. However, at the beginning of the second outer loop, the temperature of the updated first guess had been lowered sufficiently that nearly all of the observations previously rejected were now able to pass the gross-error check (see Figs. 9e and 9f).
b. Wind analysis during a high-impact event in the mid-Atlantic region
Mesonet winds provide an extraordinarily valuable source of observations. However, their usage in an assimilation must be undertaken with caution due to numerous quality control challenges. It is noted that it was not until the year 2007 that the RUC system and the NCEP North America Data Assimilation System (NDAS) began to assimilate this data type on account of the quality control challenges (Benjamin et al. 2010). Furthermore, the Global Forecast System (GFS; Sela 1980, 1982), which is one of NOAA’s most widely used atmospheric data assimilation–forecast systems, does not yet make use of these data.
The challenges associated with the assimilation of Mesonet winds during a high-impact event are illustrated by the 31 December 2008 mid-Atlantic region windstorm. Power outages and scattered damage were reported during the storm, which was associated with a strong pressure gradient. At 1500 UTC on that day, Fig. 10a displays the “non-Mesonet” winds that were available to the assimilation. The observations in Fig. 10a originate from 91 stations, of which three are surface marine stations, and the remaining 88 are METAR stations. It is noted that the METAR network is the most trusted of all networks available to the RTMA thanks to its accuracy and consistency. The reader is referred to NOAA (2005, chapter 3) for the quality control standards for METARs in the United States. Where available, the METAR winds can therefore be considered to be a good representation of the true winds. The eastern half of Virginia, as well as Maryland and the District of Columbia, are dominated by wind speeds stronger than 10 m s−1. There are only a few exceptions, with reports between 5 and 8 m s−1. The RTMA first guess, which is displayed in Fig. 10b, clearly underestimates the wind speed in the above region of focus. With the exception of small patches in central Virginia, and northeastern Virginia along the border with Maryland, where the winds are between 8 and 10 m s−1, the whole area is dominated by lighter winds. Figure 10c displays all Mesonet wind speeds faster than 8 m s−1. The picture agrees well with, and complements, that of Fig. 10a, which is composed of the most reliable stations. However, a highly contradicting picture is seen in Fig. 10d, which depicts all Mesonet winds with values lower than 8 m s−1. Figure 10c and 10d, combined, reveal that the Mesonets winds are dominated by slow-biased reports. Most of the stations report values below 8 m s−1, and in fact many of them report between 0 and 4 m s−1 in the region of focus. It is therefore clear that quality control will play a crucial role in determining the quality of the analysis for this case. Figure 10e shows the RTMA analysis when the only active layers of quality control are the MADIS flags and the GSI internal gross-error check. Just as with the first guess, Fig. 10e reveals an analysis that is dominated by winds lighter than those reported by the trusted stations in Fig. 10a. Some improvements are however present in southern, east-central, and southeastern Virginia. Finally, Fig. 10f displays the analysis when all quality control layers are activated, namely, the MADIS flags, the gross-error check, the lists of trusted providers and trusted stations, the static blacklist, and the dynamic blacklist. Although not perfect, the analysis shown in Figure 10f represents a large improvement upon the first guess, with winds above 9 m s−1 for most of the region of focus. Notice the improvement in the Washington, D.C., area and in most areas of Maryland and Virginia. However, the need to continue to improve the quality control procedure also becomes apparent. For example, although less severe than its counterpart in Fig. 10e, the “bull’s-eye” of light wind speed on the eastern shore of the Chesapeake Bay stems from a shortcoming in the quality control process. Since a tighter gross-error check could have the adverse effect of rejecting some of the good observations, further improvements to the quality control must be achieved mainly by improving the list of “trusted stations” and the dynamic blacklist.
(a) “Non-Mesonet” wind speed observations that were available for the 1500 UTC 31 Dec 2008 analysis. They included 88 from METAR reports and 3 from surface marine observing platforms. (b) First-guess wind speed in m s−1. Mesonet winds with values (c) ≥8 and (d) <8 m s−1. (e) Wind analysis when the only quality-control mechanisms used were the MADIS quality control flags and the GSI-internal gross-error check, and (f) analysis when all of the quality-control mechanisms were turned on. The colors of the small circles depicting the observations in (a),(c), and (d) follow the color code shown at the bottom of the figure.
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
c. Dewpoint “analysis” example for southern California
As mentioned in section 2, the RTMA uses the 2DVAR gridded fields of 2-m specific humidity and surface pressure to provide a diagnostic 2-m dewpoint field. Field forecasters in general have a preference for dewpoint rather than specific humidity for their routine operations.
Figure 11 illustrates how the RTMA is able to capture finescale details evident in the observations in a case when the first guess is already fairly good. Figure 11 refers to the 1800 UTC 28 January 2010 analysis, upon converting the original first guess, analysis, analysis increments, and observations expressed in terms of specific humidity into the equivalent dewpoint quantities. Figure 11a shows the first-guess dewpoint, and Fig. 11b displays the RTMA analysis. Figure 11c and 11d show the analysis increments and the observations, respectively. A comparison of Figs. 11a and 11d reveals the first guess in general reflects the observations well. A closer inspection, however, shows finescale details in the observations that are not present in the first guess. Some of these details are highlighted by the three black circles in Fig. 11a. The dewpoint values inside the northernmost and middle focus circles show a moister first guess than is revealed by the observations. Most observations report between 0° and 4°C in those two areas, while the first guess shows dewpoint values of 6°C and higher. In contrast, the values inside the southernmost circle show a first guess that is slightly drier than the observations. Some of the observations report over 10°C, while the first guess shows values below 10°C. Comparing the analysis in Fig. 11b with the first guess and the observations, it becomes apparent that the RTMA is able to correct the first guess in the above circled areas to bring it closer to the observations. Negative dewpoint increments are applied inside the northernmost and middle focus circles, while a positive increment is applied inside the southernmost circle, as revealed by Fig. 11c. It should also be noted that, just as desired, the derived dewpoint analysis increments also follow the contour lines of the underlying topography to some extent.
(a) First-guess dewpoint over southern CA at 1800 UTC 28 Jan 2010 and (b) the RTMA analysis. (c) Analysis increments as shaded contours and topography as black contours. (d) Dewpoint observations. Dewpoint units are °C in all four panels. The terrain in (c) is displayed at 300-m intervals. The actual values of the contour lines are of no consequence. The colors for the observations in (b) follow the color code shown at the bottom of the panel.
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
d. Average cross-validation statistics
In applications of data assimilation for the sole purpose of generating gridded fields of weather elements, cross-validation techniques represent perhaps the only robust methods for verifying the analysis. Cross validation denotes the use of a subset of the observed data to verify the analysis performed with these selected data withheld. The reliability and robustness of the cross-validation method can be improved by using multiple disjoint validation subsets to produce an aggregate validation statistic. It is desirable that each validation subset contains representative data from all of the geographical regions observed but without the redundancy of close pairs or tight clusters, even if these are present in the original dataset. For RTMA applications, the desired properties of representativeness and nonredundancy are achieved fairly well and efficiently with the help of a simple application of a continuous space-filling Hilbert curve (De Pondeca et al. 2006; Purser et al. 2009; Tyndall et al. 2010). All the data are mapped and sorted in the order of the curve’s single real parameter to simultaneously construct multiple disjoint subsets. It is worth noting that a commonly used alternative approach was adopted by Horel and Dong (2010). They performed similar cross-validation experiments in which all observations used in a control analysis were used except for one. Then, this leave-one-out cross-validation procedure was repeated, removing the next, etc.
The aggregate statistics in this work represent averages over five validation subsets, each containing approximately 10% of the observations. Separate subsets are constructed for each observation category (i.e., temperature, specific humidity, wind, and surface pressure). For temperature, Fig. 12 illustrates the fifth subset constructed from the pool of observations shown in Fig. 7. It displays good geographical representativeness (better than the original data) and keeps the occurrence of close pairs of observations and clustering to an acceptable minimum.
The fifth cross-validation subset computed from the universe of temperature stations shown in Fig. 7. The subset contains 1315 stations.
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
Cross-validation root-mean-square errors (cv-RMSEs) were computed for a period of 15 days from 0000 UTC 15 November 2009 through 2300 UTC 29 November 2009 at 3-h intervals. Figure 13 displays the results for the analyzed 2-m temperature, 2-m specific humidity, surface pressure, and 10-m wind speed, and the respective backgrounds. They represent average values for the RTMA domain, computed as a function of time of day. Compared with the background, Fig. 13 reveals a consistent decrease of the cv-RMSEs in the analysis for all four fields. The 15-day time-averaged statistics are summarized in Table 3. In particular, the percent improvements are 34% for temperature, 45% for specific humidity, 29% for surface pressure and 16% for wind.
Average cross-validation RMSE statistics for the 15-day period from 0000 UTC 15 Nov through 2300 UTC 29 Nov 2009 computed at 3-h intervals for (a) 2-m temperature, (b) 2-m specific humidity, (c) surface pressure, and (d) 10-m wind speed. Lines with open (closed) circles show the errors in the first guess (analysis). The statistics represent averages over five disjoint validation sets, each containing approximately 10% of the observations.
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
Time- and domain-averaged cross-validation RMSEs for the 15-day period from 0000 UTC 15 Nov through 2300 UTC 29 Nov 2009. Results are shown for the first guess (FG) and the analysis (ANL), for the fields of 2-m temperature (T), 2-m specific humidity (SH), surface pressure (Psfc), and wind speed (WSP). The percent improvement for the RMSE is obtained as IMPROV = 100(FG − ANL)/ANL.
In interpreting the cross-validation error statistics, it is important to bear in mind that they include not only the errors in the analysis, but also the measurement and representativeness errors from the cross-validating observations themselves. One implication is that the cv-RMSE tends to overestimate the true error in the analysis. Conversely, the cross-validation percentage improvement tends to underestimate the true percentage improvement in the analysis errors. Finally, it should also be pointed out that the statistics remain weighted to the data-dense regions of the central and eastern United States, simply because there are more stations in those regions. Scores are likely lower in regions of complex terrain.
6. RTMA precipitation and sky cover grids
The RTMA quantitative precipitation amount is obtained by bilinearly interpolating the so-called early version of the NCEP stage II multisensor hourly precipitation analysis from the 4-km Hydrologic Rainfall Analysis Project grid (Lin and Mitchell 2005) to the 5-km NDFD grid. A snapshot of the analysis is shown in Fig. 14. The stage II analysis is made from radar hourly precipitation estimates and hourly rain gauge data received at NCEP. The real-time radar data are from the real-time digital precipitation arrays produced at the approximately 150 CONUS Weather Surveillance Radar-1988 Doppler (WSR-88D) sites. The early version of the stage II is run at 35 min after each hour, using the approximately 1500 Automated Surface Observing System (ASOS) sites obtained from the METAR files, which are usually available within 30 min after the hour. For the benefit of clarity, the existence of the so-called mid and late versions of the stage II data is also noted. They are run at NCEP 6 and 18 h later, using the more complete set of approximately 5000 gauge reports from the Hydrometeorological Automated Data System, most of which are not received in time for the RTMA.
The RTMA 1-h accumulated precipitation (mm) valid at 2100 UTC 13 Jun 2007.
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
The RTMA sky cover represents a bilinear interpolation of the 10-km resolution NESDIS effective cloud amount (ECA) from GOES sounders (e.g., Schreiner et al. 1993; Bayler et al. 2000) to the 5-km NDFD grid. It should be noted that the ECA is a proxy for the actual sky cover as defined in NOAA (2005, chapter 9). The ECA measures the opacity of the cloud. A 100% ECA value refers to a cloud that is totally opaque to radiation from below such that none reaches the satellite sensor. In contrast, an ECA value of 10% can result from thick clouds covering 10% of the satellite field of view (FOV), or a thin cloud covering the entire FOV. The cloud algorithm accurately detects the presence of a cloud better than 95% of the time. Clear FOVs (0% ECA) are falsely detected (cloud missed) less than 5% of the time. The accuracy is affected by the season and time of the day. It tends to be closer to 95% in winter, and closer to 98% in summer. Also, the detection accuracy is better at local noon than at local sunrise. The algorithm is designed to minimize the occurrence of false detection, although on rare occasions, warm low clouds and very thin high clouds are incorrectly detected.
Figure 15 shows an example of the RTMA ECA for a selected analysis time.
The RTMA effective cloud amount (%) valid at 2100 UTC 30 Jun 2010.
Citation: Weather and Forecasting 26, 5; 10.1175/WAF-D-10-05037.1
7. Planned system improvements and ongoing work
a. Planned system improvements
The following new features have been tested and are planned for implementation during the next operational RTMA upgrade.
The nominal resolution of the CONUS-RTMA is to be refined from 5 to 2.5 km.
The so-called first guess at the appropriate time (FGAT) is to be added. In this scheme, the observation increment is computed at the exact report time for the observation by interpolating between several input first-guess fields valid at different times within the assimilation time window. The interpolated observation increment is then assumed valid at the analysis time. With this procedure in place, the assimilation time window is to be expanded from ±12 to ±30 min for all observation types, so that more data can be used in the assimilation.
A bias correction scheme for the first-guess temperature adapted from the unbiased sequential analysis algorithm of Dee and da Silva (1998) and Dee and Todling (2000) is to be added.
The ability to blend the first-guess winds from the RUC-based first guess with the winds from the Hurricane Weather Research and Forecasting Model (HWRF; Tallapragada et al. 2008) is to be added to provide a much-needed, improved first guess for tropical systems (Manikin and Pondeca 2009).
b. Ongoing work
Current work includes the following.
Testing the assimilation of low-orbit satellite winds, and of ocean surface winds derived from the satellite-borne Advanced Scatterometer (ASCAT; Gelsthorpe et al. 2000) and the multifrequency polarimetric microwave radiometer (WindSat; Gaiser et al. 2004).
Exploring the use of time-dependent autocovariances built from background field diagnostics, such as background potential temperature.
Adding the capability to analyze more meteorological parameters in the RTMA, including wind gusts (Zhu et al. 2009), planetary boundary layer height, maximum and minimum temperatures, and mean sea level pressure.
Improving the estimate of the analysis uncertainty to remove the ambiguity in the amplitude and at the same time incorporate information about the systematic errors from the background and the observations. The approach consists of using the statistics gained from routine cross-validation experiments to tune the numerical values of the error reduction computed from the Lanczos method.
Continuing to improve the quality control for Mesonet winds. The recently proposed building of the “National Mesonet of Mesonets” is expected to greatly aid in the quality control process thanks to the detailed metadata information that is to be collected along with the observations themselves. In particular, one can envisage an adaptive quality control for Mesonet winds that takes into account maps of the obstacles to the flow. It must be stressed that the advent of the National Mesonet of Mesonets is a response to the recommendations from the National Academy of Science (National Research Council 2009) for the need to improve coordination among the government agencies, commercial firms, and educational institutions that provide access to near-surface observational data across the United States.
Testing a nonlinear quality control for the observations.
Using aggregates of RUC model forecasts to derive more appropriate, location-dependent RTMA background error statistics, and applying the cross-validation technique to aid with their fine-tuning. For example, Tyndall et al. (2010) incorporate cross validation in their estimates of the background error variance of the RTMA temperature background fields as well as the observation error variance as a function of observation type (their Table 1). Their results suggest that using an observation to background error variance ratio greater than one may be beneficial. Cross validation is also a useful tool to help minimize the occurrence of analysis overfitting, which has been seen occasionally in the RTMA.
Producing a “quality mark” array accompanying each precipitation RTMA analysis, with information such as whether a grid point is outside of an effective radar coverage area, radar beam height, gauge density, etc., to give users some measure of the reliability of the analysis at each grid point.
8. Conclusions
A description of the NCEP Real-Time Mesoscale Analysis (RTMA), applied to the CONUS NDFD grid, was presented. The system applies the 2DVAR technique to analyze near-surface observations and produce 5-km-resolution gridded fields of 2-m temperature, 2-m specific humidity, 10-m U and V components of the wind, and surface pressure. In addition, it computes a diagnostic field of dewpoint temperature. The background error autocovariances of the 2DVAR are prescribed to follow the contour lines of the terrain field to a controlled degree. The 2DVAR uses the 13-km 1-h forecast from the RUC downscaled to the 5-km-resolution RTMA domain as its first guess. Along with the analysis, the 2DVAR computes gridded fields of the analysis uncertainty, which are equal to the Lanczos-based estimate of the square root of the analysis error variances. The analysis is performed hourly using observations from METARs, and synoptic, Mesonet, and marine stations. Besides honoring the MADIS quality control flags, the RTMA quality control for the observations consists of the GSI-internal gross-error check, and the use of station blacklists. For Mesonet winds, it also uses a list of trusted providers, as well as a list of trusted stations. The RTMA also creates grids of quantitative precipitation amount and sky cover by remapping the NCEP stage II precipitation data and the NESDIS GOES sounder effective cloud amount to the 5-km grid.
Three examples were presented to illustrate the performance of the RTMA-2DVAR. In the first example, the system was shown to capture a temperature cold pool in the Washington, D.C., area, which was present in the observations but absent from the first guess. The second example demonstrated the critical importance quality control plays for Mesonet winds, a data type known to exhibit a slow bias compared with the first guess. Finally, focusing on southern California, the third example demonstrated the RTMA capability to capture finescale details in the dewpoint that were present in the observations but absent from an otherwise quite reasonable first guess. Domain-averaged, cross-validation statistics were also presented for a 15-day period in November 2009. Measured in terms of the RMSE, the analysis was shown to be a significant improvement upon the background. A list of system improvements that are scheduled for implementation during the next RTMA upgrade cycle was presented. Of note is the planned doubling of the resolution to 2.5 km. The ongoing work, including testing the use of wind observations from the ASCAT and WindSat observing platforms and improving the estimate of the analysis uncertainty with the help of cross-validation statistics was described.
A main goal of this project is to see RTMA become a basic resource tool for NWS forecasters to use in their routine operations, and an important data resource serving the academic community and other federal users. Forecasters are thus expected to continue to send their feedback as the various changes are tested, and the broader community of users is urged to follow suit. Indeed, the continued system evaluation and broadening of the pool of evaluators is crucial to guaranteeing the quality of the RTMA and its multiphased upgrading toward the Reanalysis of Record.
Acknowledgments
We thank Drs. Joshua Watson, Dave Radell, and Steven Lazarus for their many contributions to this project. This research is, in part, in response to requirements and funding by the Federal Aviation Administration (FAA). The views expressed are those of the authors and do not necessarily represent the official policy or position of the FAA.
APPENDIX A
The 2DVAR Autocovariance Model of Background Errors
The first term on the right-hand side of Eq. (A2) represents the inverse isotropic aspect tensor, where Lh is the horizontal spatial correlation scale and 
APPENDIX B
Estimation of the Analysis Error via the Lanczos Algorithm
It can be shown that the adaptation of the Lanczos method described by Fisher and Courtier (1995) to take into account the 
- Compose the tridiagonal matrix
where k is the minimization inner-loop counter, δk+1 ≡ (1/αk+1 + γk/αk), and βk+1 ≡ −ck/(αkck+1). Here, αk is the minimization step size,
Find the eigenvectors and eigenvalues {φk, λk} of
- Compute the estimate of the analysis error covariance matrix aswhere
Since the number of leading vectors 
REFERENCES
Bayler, G. M., Aune R. M. , and Raymond W. H. , 2000: NWP cloud initialization using GOES sounder data and improved modeling of nonprecipitating clouds. Mon. Wea. Rev., 128, 3911–3920.
Benjamin, S. G., and Coauthors, 2004: An hourly assimilation–forecast cycle: The RUC. Mon. Wea. Rev., 132, 495–518.
Benjamin, S. G., Brown J. M. , Manikin G. , and Mann G. , 2007: The RTMA background—Hourly downscaling of RUC data to 5-km detail. Preprints, 22nd Conf. on Weather Analysis and Forecasting/18th Conf. on Numerical Weather Prediction, Park City, UT, Amer. Meteor. Soc., 4A.6. [Available online at http://ams.confex.com/ams/pdfpapers/124825.pdf.]
Benjamin, S. G., Moninger W. R. , Sahm S. R. , and Smith T. L. , cited 2010: Mesonet wind quality monitoring allowing assimilation in the RUC and other NCEP models. [Available online at http://ams.confex.com/ams/pdfpapers/124829.pdf.]
Dee, D. P., and da Silva A. M. , 1998: Data assimilation in the presence of forecast bias. Quart. J. Roy. Meteor. Soc., 124, 269–295.
Dee, D. P., and Todling R. , 2000: Data assimilation in the presence of forecast bias: The GEOS moisture analysis. Mon. Wea. Rev., 128, 3268–3282.
De Pondeca, M. S. F. V., Park S.-Y. , Purser J. , and DiMego G. , 2006: Applications of Hilbert curves to the selection of subsets of spatially inhomogeneous observational data for cross-validation and to the construction of super-observations. Preprints, AGU Fall Meeting, San Francisco, CA, Amer. Geophys. Union, A31A-0868.
Derber, J. C., and Rosati A. , 1989: A global ocean data assimilation system. J. Phys. Oceanogr., 19, 1333–1347.
Fisher, M., and Courtier P. , 1995: Estimating the covariance matrices of analysis and forecast error in variational data assimilation. ECMWF Tech. Memo. 220, 29 pp.
Gaiser, P. W., and Coauthors, 2004: The WindSat spaceborne polarimetric microwave radiometer: Sensor description and early orbit performance. IEEE Trans. Geosci. Remote Sens., 42, 2347–2361.
Gelsthorpe, R. V., Schied E. , and Wilson J. J. W. , 2000: ASCAT—Metop’s advanced scatterometer. European Space Agency Bull. 102, ESA, Paris, France, 19–27. [Available online at http://www.esa.int/esapub/bulletin/bullet102/Gelsthorpe102.pdf.]
Glahn, H. R., and Ruth D. P. , 2003: The new digital forecast database of the National Weather Service. Bull. Amer. Meteor. Soc., 84, 195–201.
Horel, J., and Colman B. , 2005: Real-time and retrospective mesoscale objective analyses. Bull. Amer. Meteor. Soc., 86, 1477–1480.
Horel, J., and Dong X. , 2010: An evaluation of the distribution of Remote Automated Weather Stations (RAWS). J. Appl. Meteor. Climatol., 49, 1563–1578.
Janjić, Z. I., 2003: A nonhydrostatic model based on a new approach. Meteor. Atmos. Phys., 82, 271–285.
Janjić, Z. I., Gerrity J. P. Jr., and Ničković S. , 2001: An alternative approach to nonhydrostatic modeling. Mon. Wea. Rev., 129, 1164–1178.
Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-Year Reanalysis Project. Bull. Amer. Meteor. Soc., 77, 437–471.
Lin, Y., and Mitchell K. E. , 2005: The NCEP stage II/IV hourly precipitation analyses: development and applications. Preprints, 19th Conf. on Hydrology, San Diego, CA, Amer. Meteor. Soc., 1.2. [Available online at http://ams.confex.com/ams/pdfpapers/83847.pdf.]
Manikin, G. S., and Pondeca M. , 2009: Challenges with the Real Time Mesoscale Analysis (RTMA). Preprints, 23rd Conf. on Weather Analysis and Forecasting/19th Conf. on Numerical Weather Prediction, Omaha, NE, Amer. Meteor. Soc., 1A.1. [Available online at http://ams.confex.com/ams/pdfpapers/152991.pdf.]
Mesinger, F., and Coauthors, 2006: North American Regional Reanalysis. Bull. Amer. Meteor. Soc., 87, 343–360.
Miller, P. A., Barth M. , Benjamin L. , Artz R. , and Pendergrass W. , 2007: MADIS support for UrbaNet. Preprints, 14th Symp. on Meteorological Observation and Instrumentation/16th Conf. on Applied Climatology, San Antonio, TX, Amer. Meteor. Soc., JP2.5. [Available online at http://ams.confex.com/ams/pdfpapers/119116.pdf.]
National Research Council, 2009: Observing Weather and Climate from the Ground Up: A Nationwide Network of Networks. National Academies Press, 234 pp.
NOAA, 2005: Federal Meteorological Handbook No. 1—Surface Weather Observations and Reports. FCM-H1-2005, 104 pp. [Available online at http://www.ofcm.gov/fmh-1/fmh1.htm.]
Purser, R. J., 2005: A geometrical approach to the synthesis of smooth anisotropic covariance operators for data assimilation. NOAA/NCEP Office Note 447, 60 pp.
Purser, R. J., Wu W.-S. , Parrish D. F. , and Roberts N. M. , 2003a: Numerical aspects of the application of recursive filters to variational statistical analysis. Part I: Spatially homogeneous and isotropic Gaussian covariances. Mon. Wea. Rev., 131, 1524–1535.
Purser, R. J., Wu W.-S. , Parrish D. F. , and Roberts N. M. , 2003b: Numerical aspects of the application of recursive filters to variational statistical analysis. Part II: Spatially inhomogeneous and anisotropic general covariances. Mon. Wea. Rev., 131, 1536–1548.
Purser, R. J., De Pondeca M. S. F. V. , and Park S.-Y. , 2009: Construction of a Hilbert curve on the sphere with an isometric parameterization of area. NOAA/NCEP Office Note 460, 18 pp.
Riishøjgaard, L.-P., 1998: A direct way of specifying flow-dependent error correlations for meteorological analysis systems. Tellus, 50A, 42–57.
Ruth, D. P., 2002: Interactive forecast preparation—The future has come. Preprints, Interactive Symp. on the Advanced Weather Interactive Processing System (AWIPS), Orlando, FL, Amer. Meteor. Soc., 3.1. [Available online at http://ams.confex.com/ams/pdfpapers/28371.pdf.]
Schreiner, A. J., Unger D. A. , Menzel W. P. , Ellrod G. P. , Strabala K. I. , and Pellet J. L. , 1993: A comparison of ground and satellite observations of cloud cover. Bull. Amer. Meteor. Soc., 74, 1851–1861.
Seguin, W. R., 2002: AWIPS. An end-to-end look. Preprints, 18th Int. Conf. on Interactive Information and Processing Systems (IIPS) for Meteorology, Oceanography, and Hydrology, Orlando, FL, Amer. Meteor. Soc., J47–J51.
Sela, J. G., 1980: Spectral modeling at the National Meteorological Center. Mon. Wea. Rev., 108, 1279–1292.
Sela, J. G., 1982: The NMC spectral model. NOAA Tech. Rep. NWS-30, 36 pp.
Tallapragada, V., Surgi N. , Liu Q. , Kwon Y. , Tuleya R. , and Connor W. O. , 2008: Performance of the Advanced Operational HWRF Modeling System during pre-implementation testing and in real-time 2007 hurricane season. Preprints, 28th Conf. on Hurricanes and Tropical Meteorology, Orlando, FL, Amer. Meteor. Soc., 4A.5. [Available online at http://ams.confex.com/ams/pdfpapers/138066.pdf.]
Tyndall, D. P., Horel J. D. , and De Pondeca M. S. F. V. , 2010: Sensitivity of surface air temperature analyses to background and observation errors. Wea. Forecasting, 25, 852–865.
Wu, W.-S., Purser R. J. , and Parrish D. F. , 2002: Three-dimensional variational analysis with spatially inhomogeneous covariances. Mon. Wea. Rev., 130, 2905–2916.
Zhu, Y., DiMego G. , Derber J. , Pondeca M. , Manikin G. , Treadon R. , Parrish D. , and Purser J. , 2009: Wind gust speed analysis in RTMA. Preprints, 23rd Conf. on Weather Analysis and Forecasting/19th AMS Conference on Numerical Weather Prediction, Omaha, NE, Amer. Meteor. Soc., 15A.3. [Available online at http://ams.confex.com/ams/pdfpapers/152738.pdf.]
