Assimilation of Surface-Based Boundary Layer Profiler Observations during a Cool-Season Weather Event Using an Observing System Simulation Experiment. Part II: Forecast Assessment

Daniel C. Hartung Cooperative Institute for Meteorological Satellite Studies, University of Wisconsin—Madison, Madison, Wisconsin

Search for other papers by Daniel C. Hartung in
Current site
Google Scholar
PubMed
Close
,
Jason A. Otkin Cooperative Institute for Meteorological Satellite Studies, University of Wisconsin—Madison, Madison, Wisconsin

Search for other papers by Jason A. Otkin in
Current site
Google Scholar
PubMed
Close
,
Ralph A. Petersen Cooperative Institute for Meteorological Satellite Studies, University of Wisconsin—Madison, Madison, Wisconsin

Search for other papers by Ralph A. Petersen in
Current site
Google Scholar
PubMed
Close
,
David D. Turner NOAA/National Severe Storms Laboratory, Norman, Oklahoma, and Department of Atmospheric and Oceanic Sciences, University of Wisconsin—Madison, Madison, Wisconsin

Search for other papers by David D. Turner in
Current site
Google Scholar
PubMed
Close
, and
Wayne F. Feltz Cooperative Institute for Meteorological Satellite Studies, University of Wisconsin—Madison, Madison, Wisconsin

Search for other papers by Wayne F. Feltz in
Current site
Google Scholar
PubMed
Close
Full access

Abstract

In this study, atmospheric analyses obtained through assimilation of temperature, water vapor, and wind profiles from a potential network of ground-based remote sensing boundary layer profiling instruments were used to generate short-range ensemble forecasts for each assimilation experiment performed in Part I. Remote sensing systems evaluated during this study include the Doppler wind lidar (DWL), Raman lidar (RAM), microwave radiometer (MWR), and the Atmospheric Emitted Radiance Interferometer (AERI). Overall, the results show that the most accurate forecasts were achieved when mass (temperature and humidity profiles from the RAM, MWR, and/or AERI) and momentum (wind profiles from the DWL) observations were assimilated simultaneously, which is consistent with the main conclusion from Part I. For instance, the improved wind and moisture analyses obtained through assimilation of these observations contributed to more accurate forecasts of moisture flux convergence and the intensity and location of accumulated precipitation (ACPC) due to improved dynamical forcing and mesoscale boundary layer thermodynamic structure. An object-based verification tool was also used to assess the skill of the ACPC forecasts. Overall, total interest values for ACPC matched objects, along with traditional forecast skill statistics like the equitable threat score and critical success index, were most improved in the multisensor assimilation cases.

Corresponding author address: Daniel C. Hartung, Cooperative Institute for Meteorological Satellite Studies/Space Science and Engineering Center, University of Wisconsin—Madison, 1225 W. Dayton St., Madison, WI 53706.E-mail: daniel.hartung@ssec.wisc.edu

Abstract

In this study, atmospheric analyses obtained through assimilation of temperature, water vapor, and wind profiles from a potential network of ground-based remote sensing boundary layer profiling instruments were used to generate short-range ensemble forecasts for each assimilation experiment performed in Part I. Remote sensing systems evaluated during this study include the Doppler wind lidar (DWL), Raman lidar (RAM), microwave radiometer (MWR), and the Atmospheric Emitted Radiance Interferometer (AERI). Overall, the results show that the most accurate forecasts were achieved when mass (temperature and humidity profiles from the RAM, MWR, and/or AERI) and momentum (wind profiles from the DWL) observations were assimilated simultaneously, which is consistent with the main conclusion from Part I. For instance, the improved wind and moisture analyses obtained through assimilation of these observations contributed to more accurate forecasts of moisture flux convergence and the intensity and location of accumulated precipitation (ACPC) due to improved dynamical forcing and mesoscale boundary layer thermodynamic structure. An object-based verification tool was also used to assess the skill of the ACPC forecasts. Overall, total interest values for ACPC matched objects, along with traditional forecast skill statistics like the equitable threat score and critical success index, were most improved in the multisensor assimilation cases.

Corresponding author address: Daniel C. Hartung, Cooperative Institute for Meteorological Satellite Studies/Space Science and Engineering Center, University of Wisconsin—Madison, 1225 W. Dayton St., Madison, WI 53706.E-mail: daniel.hartung@ssec.wisc.edu

1. Introduction

A 2009 report issued by the National Research Council (NRC) stressed the importance of establishing a comprehensive and adaptive national strategy for surface-based observations of the planetary boundary layer (PBL). This report concluded that the current observation networks are too sparse and unevenly distributed to adequately detail the thermodynamic structure of the boundary layer at high spatial and temporal resolution, which is needed for a wide variety of applications (National Research Council 2009). In Otkin et al. (2011, hereafter Part I), a regional-scale Observing System Simulation Experiment (OSSE) was performed in which temperature, water vapor, and wind profiles from a potential network of ground-based remote sensing instruments were assimilated using an ensemble Kalman filter data assimilation system. Sensors evaluated during Part I include the Doppler wind lidar (DWL), Raman lidar (RAM), microwave radiometer (MWR), and Atmospheric Emitted Radiance Interferometer (AERI). Overall, their results demonstrated that the assimilation of high-resolution PBL profiles has the potential to improve the accuracy of atmospheric analyses used by numerical weather prediction (NWP) models. The best analyses were achieved when DWL wind observations and temperature and moisture profiles from the RAM, AERI, and/or MWR were assimilated simultaneously, which showed that both mass and momentum observations are necessary to achieve the largest improvements in analysis accuracy. This study, which is a complement to Part I, examines the impact of the PBL observations on short-range (0–12 h) model forecast skill.

In the late 1980s, a ground-breaking series of studies examined the impact of a widely dispersed wind profiler network across the contiguous United States (CONUS) on NWP forecast skill using regional OSSEs (e.g., Kuo et al. 1987; Kuo and Guo 1989). Kuo and Guo (1989) later showed that continuous assimilation of wind profiler observations using nudging achieved the best analysis when both radiosonde temperature and profiler winds were assimilated simultaneously, which demonstrated the need to complement the wind profiler network with independent temperature data. Not only did wind profiler assimilation better resolve mesoscale structures, but it was also shown to impact the divergence field, directly influencing short-range (0–12 h) precipitation forecasts (Kuo and Guo 1989). Benjamin et al. (1991) showed that simultaneously assimilating temperature and wind data from Aircraft Meteorological Data Relay (AMDAR; World Meteorological Organization 2003) equipped aircraft into an operational short-range forecast model improved flight-level (250 hPa) wind forecasts, as well as geopotential height and vorticity forecasts in the mid- and upper troposphere. Only recently has the impact of assimilating existing moisture observations such as radiosonde relative humidity, GPS precipitable water, and Tropospheric Aircraft Meteorological Data and Recording (TAMDAR; AIRDAT 2004) relative humidity been investigated (e.g., Smith et al. 2007; Benjamin et al. 2010; Moninger et al. 2010). Recent results from assimilating TAMDAR observations over the Midwest show error reductions of 0.25 m s−1, 0.4 K, and 3% for wind, temperature, and relative humidity, respectively, during the initial 3-h forecast period (Moninger et al. 2010). However, aircraft observations examined in recent work still suffer from a lack of spatial and temporal coverage over the contiguous United States, limiting them from being an optimal source of continuous temperature, moisture, and wind observations in the PBL throughout the entire diurnal cycle. The surface-based profiling networks studied in Part I examined the impact on atmospheric analyses of such a PBL profiling network that is more evenly distributed across the contiguous United States than current TAMDAR observation capabilities.

In this study, we investigate the potential impact of assimilating synthetic observations from a mesoscale high-resolution ground-based PBL profiling network on short-range NWP forecasts. We first examine the impact of adding momentum information to the conventional observations over the CONUS by including simulated DWL data in the assimilation system. This is followed by investigating the forecast impact of adding only temperature and moisture observations from the state-of-the-art RAM with conventional data. Finally, combination experiments involving both high-resolution thermodynamic profiles from the RAM and more cost-effective operational instruments such as the MWR and AERI are assimilated with DWL wind data.

Many studies have noted issues with traditional point-to-point verification techniques such as not capturing improvements in the finescale spatial structure of accumulated precipitation (ACPC) provided by higher-resolution models (e.g., Ebert and McBride 2000; Mass et al. 2002; Baldwin and Kain 2006; Weisman et al. 2008), as well as their inability to accurately reflect forecasters’ subjective interpretations of forecast accuracy (Chapman et al. 2004). As part of the Spatial Forecast Verification Methods Intercomparison Project, several new verification techniques designed to overcome some of the limitations of traditional methods were proposed, including neighborhood or fuzzy-logic-based verification, scale separation, object-based verification, and field deformation (e.g., Ahjevych et al. 2009; Davis et al. 2009; Ebert 2009; Gallus 2010). For an in-depth review of these four methods, the reader is referred to Casati et al. (2008) and Gilleland et al. (2009).

In particular, the object-based approach, which is employed during this study, is able to compare and verify a variety of attributes of matched forecast and observed objects such as size, shape, and intensity (Davis et al. 2009; Ebert and Gallus 2009; Gallus 2010). One object-based verification method, known as the Method for Object-based Diagnostic Evaluation (MODE; Davis et al. 2006a,b), strives to mimic the process that a human expert would use to identify a forecast feature and then determine the level of similarity between this feature and a corresponding observed feature (Davis et al. 2009). Recent work by Gallus (2010) has shown that using a probability threshold to define objects in ensemble forecasts was a valid forecasting approach with MODE. This object-based verification technique was employed in this study, along with more traditional skill statistics, to assess 6-h ACPC associated with a synoptic-scale cool-season event.

The paper is structured as follows. Section 2 briefly describes the configuration of the high-resolution “truth” simulation and each assimilation experiment. A synoptic overview of the event is given in section 3, with an assessment of vertical mean error profiles, 6-h ACPC, and moisture flux convergence forecasts provided in section 4. A discussion and concluding remarks are given in section 5.

2. Data and methodology

a. Forecast model

Version 3.0.1.1 of the Weather Research and Forecasting (WRF) model was used for this study. WRF is a sophisticated NWP model that solves the compressible nonhydrostatic Euler equations cast in flux form on a mass-based terrain-following vertical coordinate system. Prognostic variables include the horizontal and vertical wind components, various microphysical and thermodynamic parameters, and the perturbation potential temperature, geopotential, and surface pressure of dry air. The reader is referred to Skamarock et al. (2005) for a complete description of the WRF modeling system.

b. Data assimilation system and simulated observations

In Part I of this study, a series of assimilation experiments were conducted using the ensemble Kalman filter (EnKF) algorithm implemented in the Data Assimilation Research Testbed system developed at the National Center for Atmospheric Research (Anderson et al. 2009). Simulated observations were generated for three conventional observing systems and four potential ground-based PBL profiling networks using data from a high-resolution truth simulation. Conventional observations include those from the Automated Surface Observing System (ASOS), the Aircraft Communications Addressing and Reporting System (ACARS), and radiosondes. Simulated profiler observations were created to emulate DWL, RAM, MWR, and AERI sensors. The profilers were positioned at current Weather Surveillance Radar-1988 Doppler (WSR-88D) locations in order to potentially leverage existing infrastructure and personnel resources; resulting in a network of 140 stations distributed across the United States.

c. Assimilation experiments and generation of forecasts

Atmospheric analyses obtained after 24 h of assimilation were used to initialize 12-h ensemble forecasts for the 7 assimilation experiments described in Part I. Table 1 lists the observations assimilated during each experiment. Simulated conventional observations were the only observations assimilated during the CONV case; however, these observations were also assimilated during the other cases. DWL wind observations were assimilated during the CONV-DWL case, RAM temperature and water vapor mixing ratio observations during the CONV-RAM case, and DWL and RAM observations during the CONV-RD case. The DWL wind observations were then assimilated with temperature and moisture profiles from the MWR and AERI during the CONV-MD and CONV-AD cases, respectively. All of the AERI, MWR, and DWL observations were assimilated during the CONV-MAD case (Table 1). Finally, 12-h ensemble forecasts were subsequently generated for each case using the final analyses from 0000 UTC 8 January 2008. Figure 1 shows the full procedure of the OSSE for this study in graphical form, while Fig. 2 highlights the subdomain used for the forecast verification.

Table 1.

Assimilation experiments, with respective simulated observations, included in the forecast analyses.

Table 1.
Fig. 1.
Fig. 1.

Conceptual model for the OSSE. Step 1 involved the generation of the initial ensemble that is integrated forward in time 24 h. Conventional observations are assimilated during step 2 with both conventional and ground-based profiler observations assimilated in step 3. Finally, 12-h forecasts are performed during step 4.

Citation: Monthly Weather Review 139, 8; 10.1175/2011MWR3623.1

Fig. 2.
Fig. 2.

WRF model domain for all forecast runs (terrain elevation in meters above mean sea level shaded) with the forecast statistical verification subdomain outlined with a solid white line.

Citation: Monthly Weather Review 139, 8; 10.1175/2011MWR3623.1

3. Truth simulation

The configuration of the high-resolution truth simulation tracking the evolution of several extratropical weather systems across the contiguous United States is described in Part I. Here, we present the evolution of the simulated surface and 500-hPa conditions from the truth simulation spanning the forecast period from 0000 UTC to 1200 UTC 8 January. At 0000 UTC, a broad upper-level trough containing a pair of embedded shortwaves was located over the central Rocky Mountains with a strong jet streak (>60 m s−1) extending from the southern plains to the Great Lakes (Fig. 3a). An intense surface baroclinic zone extended from the Oklahoma Panhandle northeast to southern Ontario. Strong low-level southerly flow was transporting very moist air northward from the Gulf of Mexico to central Missouri with mixing ratios approaching 9 g kg−1 as far north as Montreal, Quebec, Canada (Fig. 3b). By 0600 UTC, the 500-hPa trough had progressed slightly to the east and the amplitude of the embedded shortwaves increased as the system encountered a dominant quasi-stationary ridge over the mid-Atlantic region resulting in a slight intensification of the jet streak over the upper Midwest (Fig. 3c). This eastward progression of the longwave trough juxtaposed the upper-level jet entrance region with an area of surface cyclogenesis along the baroclinic zone over northern Oklahoma at that time (not shown). The elongated surface baroclinic zone over the central United States intensified while the southern portion of the front had progressed to the southeast over central Oklahoma and the northern portion remained nearly stationary over Lake Michigan. Persistent southerly flow (~25 m s−1) continued to advect moisture northward to the east of the front over the lower and mid-Mississippi River valley (Fig. 3d). The 500-hPa trough deepened farther as it progressed east over the central plains by 1200 UTC (Fig. 3e). Previous short waves emanating from the cyclonic shear side of the jet downstream of the trough axis at 0600 UTC had been absorbed in confluent flow and were no longer evident (Fig. 3e). The surface cyclone rapidly deepened over the Great Lakes while the surface frontal zone remained nearly stationary over the Midwest region with small-scale increases in the temperature gradient collocated with wavelike disturbances that developed over central Oklahoma and along the Iowa–Illinois border (Fig. 3f). Finally, a narrow corridor of warm moist air continued to stream northward over the eastern Great Lakes with mixing ratio values approaching 14 g kg−1 along the Gulf Coast (Fig. 3f).

Fig. 3.
Fig. 3.

(a) Simulated 500-hPa geopotential height (shaded every 20 m) and winds (m s−1) valid at 0000 UTC 8 Jan 2008. (b) Simulated 100-m AGL water vapor mixing ratio (g kg−1; shaded), winds (m s−1), and temperature (contoured in black every 2°C) valid at 0000 UTC 8 Jan 2008. (c),(d),(e),(f) As in (a),(b), but valid at 0600 UTC 8 Jan and 1200 UTC 8 Jan 2008, respectively. Dashed white lines denote the location of trough axes.

Citation: Monthly Weather Review 139, 8; 10.1175/2011MWR3623.1

4. Results

a. Mean error profiles

As a first step in assessing the potential forecast impact of a multisensor surface-based observing network, vertical profiles of root-mean-square error (RMSE) and bias reduction for each assimilation case are shown in Figs. 4 and 5. The difference profiles were computed for each assimilation case using data from the ensemble mean within the forecast verification subdomain (refer to Fig. 2) and then subtracting the resultant CONV error profiles from the profiles for the other cases. In agreement with Part I of this study, the profiler assimilation experiments generally contain smaller errors than the CONV case throughout the forecast period, with the vertical extent and magnitude of the error growth dependent on which observations are assimilated.

Fig. 4.
Fig. 4.

Vertical profiles of (a) root-mean-square error reduction and (b) difference in bias [right side of (a) and (b)] for temperature (K) computed by subtracting the error profile for a given case from the CONV profile shown on the left side of (a) and (b). The profiles were computed using data from the 0-h analysis ensemble mean valid at 0000 UTC 8 Jan in the forecast verification region shown in Fig. 2. (c),(d) As in (a),(b), but for relative humidity (%). (e),(f) As in (a),(b), but for wind vector speed (m s−1). Results are shown for the CONV-DWL (dashed green), CONV-RAM (dashed blue), CONV-RD (dashed red), CONV-MD (blue), CONV-AD (green), and CONV-MAD (red) experiments.

Citation: Monthly Weather Review 139, 8; 10.1175/2011MWR3623.1

Fig. 5.
Fig. 5.

As in Fig. 4, but for the 6-h forecast valid at 0600 UTC 8 Jan 2008. All statistics were computed using the forecast ensemble mean.

Citation: Monthly Weather Review 139, 8; 10.1175/2011MWR3623.1

Similar to the results described in Part I for the full CONUS domain, the 0-h temperature and moisture analyses valid at 0000 UTC 8 January for the forecast subdomain were degraded in the mid- to lower troposphere during the CONV-DWL case (Figs. 4a–d). Assimilation of temperature and moisture observations from the AERI and MWR reduced the RMSE and bias in the lower troposphere, most notably in the CONV-AD and CONV-MAD cases with their overall impact diminishing above 600 hPa as the error profiles slowly converge toward the CONV profile (Figs. 4a–d). Assimilation of RAM observations during the CONV-RAM and CONV-RD cases produced the smallest temperature and moisture errors at the final analysis time, especially in the mid- and upper troposphere where the substantial positive impact of nocturnal thermodynamic profiles from the Raman lidar is evident (Figs. 4a–d).

Contrary to the positive impacts on the thermodynamic fields, assimilation of RAM profiles alone degraded the vector wind analysis below 400 hPa in the CONV-RAM case (Figs. 4e,f). Minor differences exist below 400 hPa between cases in which DWL wind observations are assimilated, supporting the conclusion from Part I that the improved wind accuracy relative to the CONV case is due to the DWL wind observations. However, as an indirect affect on the wind field through mass balance constraints, high-quality nocturnal RAM observations near the tropopause had a larger positive impact on the upper-tropospheric wind field in the CONV-RAM and CONV-RD cases compared to the AERI and MWR (Figs. 4e,f).

By 0600 UTC, the CONV-DWL case had larger forecast errors below 700 hPa than the CONV case, while all other experiments were characterized by smaller temperature and humidity forecast errors above that level (Figs. 5a–d). Forecast errors for temperature and moisture were smaller during the CONV-AD, CONV-MD, and CONV-MAD cases, with the AERI having a slightly greater impact than the MWR (Figs. 5a–d). Compared to the CONV-MD case, the smaller errors during the CONV-MAD and CONV-AD cases are attributed to the greater number and larger information content of the AERI observations. The largest temperature and humidity forecast error reductions occurred throughout the entire troposphere during the CONV-RAM and CONV-RD cases (Figs. 5a–d); however, the positive impact of the RAM observations above 400 hPa is diminished compared to the final analysis at 0000 UTC (Figs. 5a–d). Examination of vector wind error profiles (Figs. 5e,f) shows that cases in which temperature and moisture profiles from the AERI and MWR are included with DWL winds (CONV-AD, CONV-MD, and CONV-MAD) had the smallest errors below 400 hPa (Figs. 5e,f). Finally, similar forecast errors for temperature, humidity, and vector wind below 750 hPa occurred during the CONV-AD, CONV-MAD, and CONV-RD cases, which suggests for this particular case study that combinations of the commercially available AERI and MWR instruments may be capable of adding similar value to short-range PBL forecasts as observations from the research grade RAM (Figs. 5e,f).

b. 6-h accumulated precipitation forecasts

Next, we compare the ACPC from the different assimilation experiments to that observed in the truth simulation. Figure 6 shows 6-h ACPC with 850-hPa equivalent potential temperature θe contours and 4-km above ground level (AGL) wind vectors valid at 0600 and 1200 UTC 8 January for the truth simulation and each assimilation case. During the first 6-h forecast period, a narrow band of light to moderate ACPC developed in the truth simulation ahead of the low-level baroclinic zone within the θe axis extending from southwest Missouri to Lake Erie (Fig. 6a). Inspection of the CONV case (Fig. 6c) shows that the southern flank of the thermodynamic gradient extending from southern Texas to the Kansas–Oklahoma border is much weaker than indicated by the truth simulation, with the two embedded wavelike disturbances over Kansas and southeast Minnesota unresolved (Figs. 6a,c). The midlevel trough axis, diagnosed by the 4-km wind field, is broader and shifted approximately 150 km to the east of truth with slightly weaker wind speeds (Figs. 6a,c). Also, only one ACPC maximum is present and is larger and displaced too far to the north over east-central Michigan and Lake Huron (Fig. 6c).

Fig. 6.
Fig. 6.

Simulated 6-h accumulated precipitation (mm; shaded), 4-km AGL wind (m s−1), and 850-hPa equivalent potential temperature (contoured in black every 4 K) valid at (a) (left) 0600 UTC 8 Jan 2008 and (b) (right) 1200 UTC 8 Jan 2008 for the truth simulation. (c)–(p) As in (a),(b), but valid for ensemble mean forecasts of (c),(d) CONV; (e),(f) CONV-DWL; (g),(h) CONV-RAM; (i),(j) CONV-RD; (k),(l) CONV-MD; (m),(n) CONV-AD; and (o),(p) CONV-MAD assimilation experiments. Red dashed lines indicate the position of the trough axis at 4 km above ground level.

Citation: Monthly Weather Review 139, 8; 10.1175/2011MWR3623.1

Assimilation of wind observations from the DWL greatly reduces the errors in the position and intensity of the midlevel trough that occurred during the CONV case; consequently, the location and intensity of the low-level baroclinic zone is also improved (Figs. 6c,e). However, since additional mesoscale temperature and moisture information was not assimilated during this case, errors in the moisture field persist. The result is a worse ACPC location forecast as sporadic ACPC develops south of Missouri, even though enhanced moisture transport from the additional wind data seems to provide a better intensity forecast than the CONV case (Fig. 6e). Including temperature and moisture data from the RAM slightly reduces the ACPC intensity errors and also minimizes the location errors by producing a more confined band of precipitation along the front (Fig. 6g). However, the additional RAM data did not improve the position or strength of the midlevel trough, and therefore the location and intensity of the surface front is slightly worse than the CONV-DWL forecast (Figs. 6c,e,g). In agreement with results presented in section 4a, assimilation of DWL wind data with the RAM, MWR, and/or AERI thermodynamic profiles had the largest reduction in position and intensity errors of the midlevel trough and surface baroclinic zone, while more accurately resolving the boundary layer mesoscale temperature and moisture structure of the cyclone (Figs. 6i,k,m). Therefore, joint assimilation of wind and thermodynamic data greatly reduced both the ACPC location and intensity errors seen in the CONV forecast (Figs. 6i,k,m). It is interesting to note that only the forecast from the CONV-MD case suggested the development of two ACPC maxima along the front extending from north-central Arkansas into northern Indiana (Fig. 6k). All other experiments with multisensor assimilation (i.e., CONV-RAM, CONV-AD, and CONV-MAD) were better able to correct the location error of the northernmost extent of the ACPC over northern Indiana, as well as limit the ACPC intensity error within the band (ACPC maxima >20 mm; Figs. 6i,m,o).

During the second 6-h forecast period ending at 1200 UTC 8 January, heavy precipitation in excess of 40 mm was observed from the Ozarks to the southern Great Lakes in the truth simulation (Fig. 6b). The southern portion of the frontal boundary over Texas nearly doubled in intensity from the previous period with ~40 m s−1 southwesterly flow parallel to the surface front at 4-km AGL, which suggests conditions were favorable for training convection and heavy ACPC. Once again, as a consequence of a weaker large-scale trough and eastward displacement, conventional observations alone produced a low-level baroclinic zone that was too weak and located too far to the east by this time (Fig. 6d). Weaker winds aloft decreased the northward moisture transport and contributed to the eastward displacement of a broad, diffuse region of precipitation ahead of the front (Fig. 6d). Similar to the 0600 UTC forecast period, errors in precipitation intensity are reduced when DWL winds are assimilated, but only a small improvement in ACPC location errors is discernable (Fig. 6f). Assimilating RAM temperature and moisture profiles reduced errors in the BL thermodynamic structure compared to the CONV case (Fig. 6h). This produced a stronger baroclinic zone positioned farther to the west than the CONV case, which substantially reduced the position error in the precipitation band (Fig. 6h). Large errors in the upper-level winds were still present, however, leading to continued underrepresentation of the moisture transport and precipitation intensity. Finally, when DWL winds were assimilated with temperature and moisture profiles from the RAM, MWR, and/or AERI, substantial error reductions were observed in both precipitation location and intensity (Figs. 6j,l,n,p). The greatest improvements in ACPC forecast intensity occurred during the CONV-RD, CONV-MD, and CONV-AD cases, while assimilating AERI and MWR profiles with DWL winds more accurately resolved the bimodal ACPC maxima over Missouri and Illinois (Figs. 6l,n,p). All multisensor cases were able to more accurately forecast the intensity of the surface front at 1200 UTC compared to CONV; however, the forecasted intensity of the frontal zone was still weaker than observed with less moisture transport and lower 6-h ACPC totals (e.g., Figs. 6b,j). Overall, the better ACPC intensity and location forecasts during the joint assimilation cases are due to the more accurate representation of the wind and moisture fields (refer to Fig. 5).

c. 850-hPa moisture flux convergence forecasts

Forecasts of moisture flux convergence (MFC), a necessary condition for heavy precipitation, are shown in Fig. 7. The narrow band of moderate ACPC during the first forecast period (Fig. 6a) corresponds well with a small region of stronger MFC extending from central Missouri northeast to lower Michigan, with a local maximum in MFC over northern Indiana and southern Michigan (Fig. 7a). Similar to the 6-h ACPC forecast, the CONV case contains large eastward displacement errors in the MFC with weaker intensity, most notably over eastern Texas (Fig. 7c). Large MFC intensity errors are also present throughout the southeast United States. Assimilating DWL wind observations produced a quasi-linear MFC band compared to the CONV forecast but with slightly more accurate placement and spatial extent over the lower Great Lakes and eastern Texas (Fig. 7e). Inclusion of RAM temperature and moisture observations produces a MFC field characterized by a cellular pattern with a more focused maximum over northern Indiana (Fig. 7g). When DWL winds and thermodynamic profiles from the RAM, MWR, and/or AERI, are both assimilated, the MFC intensity and location errors are reduced over eastern Texas, with a better placement of the MFC over Michigan and Indiana compared to the CONV-DWL case (Figs. 7i,k,m).

Fig. 7.
Fig. 7.

Simulated 850-hPa moisture flux divergence (g kg−1 s−1; contoured every 0.8 × 10−4 g kg−1 s−1) and 850-hPa moisture flux vectors (kg kg−1 m s−1) valid at (a) (left) 0300 UTC 8 Jan 2008 and (b) (right) 0900 UTC 8 Jan 2008 for the truth simulation. (c)–(p) As in (a),(b), but valid for ensemble mean forecasts of (c),(d) CONV; (e),(f) CONV-DWL; (g),(h) CONV-RAM; (i),(j) CONV-RD; (k),(l) CONV-MD; (m),(n) CONV-AD; and (o),(p) CONV-MAD assimilation experiments.

Citation: Monthly Weather Review 139, 8; 10.1175/2011MWR3623.1

At the 0900 UTC forecast time, very strong MFC is observed over eastern Texas with a wedge of intense northward moisture transport over the lower Mississippi River valley culminating in a narrow concentrated line of MFC along the surface front over Missouri, Illinois, and Indiana (Fig. 7b). This is coincident with the narrow corridor characterized by 6-h ACPC in excess of 50 mm during the same forecast period (Fig. 6b). In the CONV case, the band of MFC along the surface frontal boundary is substantially weaker than truth and displaced ~200 km to the east (Fig. 7d). In the CONV-DWL case, the additional wind information helped confine the MFC over central Illinois and Indiana to a narrower banded structure but with larger errors in the intensity forecast (Fig. 7f). In contrast, assimilating RAM moisture and temperature data reduces the intensity errors over eastern Texas and begins to reorient the larger band that formed in the CONV case, but still produces larger displacement errors than the DWL case (Fig. 7h). Once again, assimilation of mass and momentum profiles produces a much better MFC intensity forecast over Texas, as well as more concentrated MFC along the surface front compared to the CONV case (Figs. 7j,l,n,p). The moisture flux vector RMSE and mean absolute error (MAE) were notably lower for all of the multisensor assimilation cases for forecasts valid at both 0300 and 0900 UTC, with the lowest RMSE and MAE observed in the CONV-MAD forecast at both times (Table 2). These results indicate that the improved MFC forecasts provided by the multisensor assimilation experiments are able to improve the ACPC forecasts.

Table 2.

850-hPa moisture flux vector (kg m kg−1 s−1) RMSE, MAE, and bias for all assimilation experiment ensemble-mean forecasts valid at 0300 UTC 8 Jan 2008 and 0900 UTC 8 Jan 2008. Boldface numbers highlight the lowest error and bias values at each of the two forecast times.

Table 2.

d. Traditional forecast statistical analysis

Figure 8a shows the equitable threat score (ETS; e.g., Gandin and Murphy 1992) for the CONV only ensemble-mean forecasts for multiple thresholds of 6-h ACPC valid at 0600 and 1200 UTC 8 January. First, there is an increase in skill between 0600 and 1200 UTC for most of the ACPC field. For the >2.4-, >6.0-, >12.0-, and >24.0-mm thresholds, skill is maximized during the 6-h period ending at 1200 UTC, in which most of the precipitation was observed (Figs. 8b,a). The zero skill for ACPC greater than 24 mm at 0600 UTC is attributed to one or more grid points with forecast ACPC > 24 mm with no verifying grid points with as large of an ACPC value in the truth observations (Fig. 8a). Similar trends appear for all intensity bins in the critical success index (CSI; Donaldson et al. 1975; Schaefer 1990; Wilks 1995; Fig. 9). Improvements in CSI are greatest in cases where DWL winds are assimilated, while the largest CSI increase in the moderate ACPC field (>6.0 mm) is evident when additional RAM temperature and mixing ratio data is assimilated (Fig. 9b). Finally, the false alarm rate is decreased by as much as 5% at all forecast times compared to the CONV case when RAM, MWR, and/or AERI profiles are assimilated (not shown).

Fig. 8.
Fig. 8.

(a) Equitable threat scores for 6-h accumulated precipitation (mm) using conventional data only valid at 0600 and 1200 UTC 8 Jan 2008 at the following thresholds: >0.20 mm (medium gray; shaded), >2.4 mm (light gray; shaded), >6.0 mm (black; shaded), >12.0 mm (dark gray; shaded), and >24.0 mm (solid white). (b) Equitable threat score differences from conventional only data for the 0600–1200 UTC 8 Jan 2008 forecast period for each 6-h accumulated precipitation intensity threshold shown in (a). Forecasts from all assimilation experiments shown in Table 1 are displayed.

Citation: Monthly Weather Review 139, 8; 10.1175/2011MWR3623.1

Fig. 9.
Fig. 9.

(a) Critical success index for 6-h accumulated precipitation (mm) using conventional data only valid at 0600 and 1200 UTC 8 Jan 2008 at the following thresholds: >0.20 mm (medium gray; shaded), >2.4 mm (light gray; shaded), >6.0 mm (black; shaded), >12.0 mm (dark gray; shaded), and >24.0 mm (solid white). (b) Critical success index differences from conventional only data for the 0600–1200 UTC 8 Jan 2008 forecast period for each 6-h accumulated precipitation intensity threshold shown in (a). Forecasts from all assimilation experiments shown in Table 1 are displayed.

Citation: Monthly Weather Review 139, 8; 10.1175/2011MWR3623.1

In addition to the 250% increase in skill when RAM temperature and moisture profiles are assimilated during the CONV-RAM case, there is a 30% increase in forecast skill in the most intense ACPC bin (>24.0 mm) in all cases when DWL winds are assimilated (Fig. 8b). Overall, the ensemble-mean forecasts from the multisensor assimilation experiments exhibit better skill than the CONV case for all intensity thresholds (Fig. 8b). Although traditional forecast skill statistics generally show the largest skill improvement for the heaviest ACPC when temperature, moisture, and wind profiles are assimilated, this is not apparent for the total ACPC field (>0.2 mm) at 0600 and 1200 UTC (Fig. 8). To better understand the improvements to the ACPC forecast skill suggested by the traditional statistics, an object-based verification approach was applied to the 6-h accumulated precipitation forecasts.

e. MODE forecast verification

The Method for Object-based Diagnostic Evaluation (MODE; Davis et al. 2006a,b) package in the Model Evaluation Toolbox (additional information available online at http://www.dtcenter.org/met/users) was used to further assess the accuracy of 6-h ACPC forecasts valid at 6 and 12 h into the forecast period. MODE is a features-based approach that provides an objective basis for forecast evaluation, while minimizing the potential for a presumably good forecast from an operational viewpoint to be penalized twice for both not being collocated with the observed field and also being located in a place other than that of the observations.

Several parameters are weighted within MODE using a fuzzy-logic approach to calculate a “total interest” value for each forecast–observation object pair; the more alike the forecast and observed objects are, the more likely the forecast object is a numerical representation of the observed member of the matched pair (Davis et al. 2009). Parameter weights to be set include those for the distance separating the centroids and boundaries of the forecast and observed objects, the distance between their convex hulls, the object orientation angle, area ratio of the objects, ratio of the objects’ intersection to the minimum area of the two objects, complexity ratio (measure of the size of the most intense precipitation with respect to the total object size), and the ratio for the 75th intensity percentile of each object. In this case, the convex hull represents the boundary of the 75th intensity percentile of the ACPC of a particular object. Because of the banded nature of the heavy precipitation in the truth simulation (Fig. 6b), the convex hull distance, complexity ratio, and intensity ratio were given a weight one increment greater than the rest of the parameters in the total interest calculation. Prior to object identification, a convolution radius (3 grid point smoother) was applied to the observed truth ACPC to better account for the lower ACPC in the ensemble-mean forecasts. In this study, an object is defined as any group of 3+ grid boxes with 6-h ACPC ≥ 6.25 mm. All forecast–observation object pairs with total interest values greater than 0.75 are matched and shown in Fig. 10 with corresponding total interest values displayed in Table 3. The reader is referred to Davis et al. (2006a) and Davis et al. (2009) for a complete explanation of the MODE object matching attributes.

Fig. 10.
Fig. 10.

MODE statistical analysis of 6-h accumulated precipitation (ACPC) (mm; shaded) objects greater than 6.25 mm ACPC valid at (left) 0600 UTC 8 Jan and (right) 1200 UTC 8 Jan 2008. (a),(b) Identify the “truth” simulation 6-h ACPC objects (contoured yellow) that are compared to the ensemble-mean forecast ACPC objects (contoured red) from the (c),(d) CONV; (e),(f) CONV-DWL; (g),(h) CONV-RAM; (i),(j) CONV-RD; (k),(l) CONV-MD; (m),(n) CONV-AD; and (o),(p) CONV-MAD assimilation experiments. Truth objects introduced in (a),(b) are also shown at their respective times in each of the assimilation forecasts to assist in interpretation. Light gray shaded region denotes the domain used for the ACPC forecast analysis.

Citation: Monthly Weather Review 139, 8; 10.1175/2011MWR3623.1

Table 3.

MODE matched forecast and observation (truth) objects in the 6-h accumulated precipitation field from Fig. 10 valid at 0600 UTC 8 Jan and 1200 UTC 8 Jan 2008. The MODE computed total interest for each matched object pair is also presented. The observed–forecast object pairs at each forecast time that share the highest total interest values are displayed in boldface.

Table 3.

During the first 6-h forecast period ending at 0600 UTC 8 January, two distinct objects were identified by MODE in the truth observations, both of which constitute a broken line extending from central Missouri northeast to the Michigan border (Fig. 10a). The CONV forecast generates a much larger and more diffuse band of precipitation extending from southeast Oklahoma to Lake Huron with 6-h ACPC greater than 6.25 mm over the entire object region, with MODE identifying the entire band as one object (Fig. 10c). The portion of the CONV ACPC band over central Missouri was somewhat closer in intensity to the truth and is in fairly good proximity to the second observed object (Figs. 10a,c). Therefore, the broad band of CONV ACPC was a slightly better match with observed object 2 compared to object 1 (Fig. 10c), but overall total interest values between both matched pairs was marginal at ~0.8 (Table 3). Assimilating DWL wind observations during the CONV-DWL case yields a slightly better precipitation intensity forecast than the CONV case; however, the large displacement error of the intensity maxima produces a substantially lower interest value than the CONV-RAM matched pair (Fig. 10e; Table 3). Total interest values for the CONV-RAM case are notably higher than the CONV-DWL and CONV cases because of the better collocation of the higher intensity ACPC maxima with object 2 in the truth ACPC field (Fig. 10g; Table 3). Forecasts made from the final analysis time when both mass and momentum profiler observations were assimilated (CONV-RD, CONV-MD, CONV-AD, and CONV-MAD cases) all show improvements in the total interest values for truth objects 1 and 2 matched pairs over conventional observations alone. The highest total interest for forecast objects matched with the first truth object in the initial 6-h ACPC field occurred when thermodynamic and DWL winds were assimilated during the CONV-MD and CONV-MAD cases (Figs. 10a,k,o; Table 3).

During the 0600–1200 UTC forecast period, convection develops and trains along the surface frontal boundary extending from southwestern Missouri northeast to northern Indiana; resulting in one large observed object (Fig. 10b). The CONV case produces a forecast that is much weaker with a more pronounced north–south orientation; however, the region of maximum ACPC in the CONV forecast is very close in proximity to the observed linear maximum across Missouri at 1200 UTC (Figs. 10b,d). The total interest for the CONV matched object pair is ~0.94 (Table 3). When assimilating high-resolution profiles from the Raman lidar (CONV-RAM), ACPC intensity decreases even though the orientation of the band is improved compared to the CONV case (Fig. 10h). Therefore the total interest of observed and CONV-RAM matched objects is slightly worse than for the CONV case due to the increased weight placed on he intensity and convex hull parameters (Table 3). When DWL observations are assimilated, the forecast intensity is much closer to the truth during the CONV-DWL case than for either the CONV or CONV-RAM cases; however, the orientation and location of the CONV-RAM ACPC object is better (Figs. 10f,h). As a result, total interest value for the CONV-DWL matched pair is slightly better than the CONV case (Table 3).

Large improvements in both location and intensity of the ACPC forecasts were made when temperature and moisture profiles (i.e., RAM, MWR, and/or AERI) and DWL wind observations were assimilated simultaneously (Figs. 10j,l,n,p). Matched object total interest values for all of these experiments markedly exceeded those of the CONV case. Finally, matched pairs between CONV-AD and CONV-MAD objects and truth yielded the highest total interest (>0.96) for the forecast period valid at 1200 UTC (Table 3) since they captured the overall location of the precipitation band, as well as the location and intensity of the ACPC double-banded maxima extending from southern Missouri northeast to northern Indiana (Figs. 10n,p).

5. Discussion and conclusions

In Part I, a regional OSSE was used to examine how the assimilation of temperature, water vapor, and horizontal wind profiles from a potential array of ground-based remote sensing PBL profiling instruments impacts the accuracy of atmospheric analyses at mesoscale resolution when assimilated using an ensemble Kalman filter data assimilation system. Simulated profiles from remote sensing systems such as the DWL, RAM, MWR, and AERI were assimilated. Overall, the best analysis was achieved when DWL wind profiles and temperature and moisture profiles from the RAM, MWR, and/or AERI were assimilated simultaneously; illustrating that both mass and momentum observations are necessary to improve the analysis. In this study, atmospheric analyses obtained after 24 h of assimilation were used to generate 12-h ensemble forecasts for each assimilation experiment described in Part I.

Overall, the results show that the most accurate forecasts were achieved when mass (temperature and humidity profiles from the RAM, MWR, and/or AERI) and momentum (wind profiles from the DWL) observations were assimilated simultaneously, which is consistent with the main conclusion from Part I. Results showed that conventional observations alone were not able to accurately predict the position and intensity of the midlevel trough, leading to the development of a weaker surface baroclinic zone displaced farther to the east than in the truth simulation.

Inclusion of DWL winds alone fails to resolve the thermodynamic structure of the surface front, leading to minimal improvement in ACPC location errors. Thermodynamic observations from the research-grade RAM greatly reduced errors in the intensity and location of the mesoscale structure of the low-level baroclinic zone, which improved the ACPC field at each forecast time; however, similar results would have been achieved if the thermodynamic observations from the commercially available MWR or AERI were used instead of the RAM. The ACPC improvements provided by the additional thermodynamic data from the RAM were likely the result of large reductions in location and intensity forecasts of low-level moisture flux. In agreement with the vertical error profiles, comparison of the ACPC and 850-hPa equivalent potential temperature fields between the truth simulation and each assimilation experiment forecast showed that assimilating DWL winds produced a better representation of the midlevel dynamic forcing (position and intensity of the trough axis) and also PBL moisture transport; therefore reducing errors in ACPC intensity through improvements in MFC forecasts. A beneficial synergy between sensors was also evident when assimilating additional mass and momentum observations simultaneously; errors in the intensity, location, and structure of the surface frontal zone were substantially reduced, which ultimately decreased precipitation location and intensity errors, with the largest intensity error reductions observed in the CONV-RD, CONV-MD, and CONV-AD cases. All of the multisensor assimilation forecasts of MFC showed large error reductions over the Gulf Coast and ahead of the low-level baroclinic zone, with the lowest RMSE and MAE observed in the CONV-MAD case at both forecast times. Therefore, the similar results between cases suggest that improvements in MFC forecasts have the potential to improve ACPC forecasts. As was the case for the assimilation analysis, factor separation techniques (e.g., Stein and Alpert 1993; Rostkier-Edelstein and Hacker 2010) could be used to more fully quantify the synergisms between observed and unobserved variables, primarily after a more diverse dataset is generated.

This study also employed the MODE object-based verification tool to assess the skill of the ACPC forecasts. Matched object pairs in the CONV-DWL and CONV-RAM cases had higher total interest values than CONV but less than the multiprofiler experiments. Matched pairs between CONV-AD and CONV-MAD objects and “truth” yielded the highest total interest (>0.96) for the forecast period valid at 1200 UTC since they captured the best overall location of the precipitation band, as well as the location and structure of the ACPC maxima. Finally, all of the assimilation experiments discussed in this study improved the equitable threat score compared to the CONV case for all ACPC thresholds during both forecast periods. Last, improvements in forecast skill for the total ACPC field (>0.2 mm) were greatest when both thermodynamic and momentum observations were assimilated in the CONV-RD, CONV-AD, and CONV-MAD cases.

In general, improvements in the ACPC intensity forecast due to better representation of moisture transport result when DWL observations are assimilated, while improvements in the location forecast occur when assimilating mass observations due to better forecasts of surface baroclinic zone location and intensity. However, assimilating additional thermodynamic observations alone did not produce strong enough moisture transport to adequately predict the heaviest precipitation during this event, nor were the additional observations able to reduce errors in the large-scale dynamic forcing of the event. The largest ACPC forecast improvements in both location and intensity were made when PBL temperature, moisture, and wind observations were assimilated simultaneously in the CONV-RD, CONV-MD, CONV-AD, and CONV-MAD cases.

Acknowledgments

We extend a sincere thank you to John Halley-Gotway and Paul Oldenburg (DTC/UCAR) for their support and assistance with the installation, configuration, and implementation of the MET and MODE software packages that were used in this study. Funding for this project was provided by the National Oceanic and Atmospheric Administration (NOAA) in support of recommendation 8a in the National Research Council (2009) report. Assimilation experiments leveraged computational resources purchased with CIMSS GOES-R project funds. Two anonymous reviewers provided thorough comments that enhanced the clarity and quality of this manuscript.

REFERENCES

  • Ahjevych, D., E. Gilleland, B. Brown, and E. Ebert, 2009: Application of spatial verification methods to idealized and NWP gridded precipitation forecasts. Wea. Forecasting, 24, 14851497.

    • Search Google Scholar
    • Export Citation
  • AIRDAT, 2004: TAMDAR sensor and system overview: Airborne real time atmospheric data collection and reporting system. AIRDAT, LLC, Morrisville, NC, 10 pp. [Available online at http://www.airdat.com/tamdar/tech_docs/TAMDAR%20Overview%20101904.pdf.]

    • Search Google Scholar
    • Export Citation
  • Anderson, J., T. Hoar, K. Raeder, H. Liu, N. Collins, R. Torn, and A. Avellano, 2009: The Data Assimilation Research Testbed: A community facility. Bull. Amer. Meteor. Soc., 90, 12831296.

    • Search Google Scholar
    • Export Citation
  • Baldwin, M. E., and J. S. Kain, 2006: Sensitivity of several performance measures to displacement error, bias, and event frequency. Wea. Forecasting, 21, 636648.

    • Search Google Scholar
    • Export Citation
  • Benjamin, S. G., K. A. Brewster, R. Brümmer, B. F. Jewett, T. W. Schlatter, T. L. Smith, and P. A. Stamus, 1991: An isentropic 3-hourly data assimilation system using ACARS aircraft observations. Mon. Wea. Rev., 119, 888906.

    • Search Google Scholar
    • Export Citation
  • Benjamin, S. G., B. D. Jamison, W. R. Moninger, S. R. Sahm, B. E. Schwartz, and T. W. Schlatter, 2010: Relative short-range forecast impact from aircraft, profiler, radiosonde, VAD, GPS-PW, METAR, and mesonet observations via the RUC hourly assimilation cycle. Mon. Wea. Rev., 138, 13191343.

    • Search Google Scholar
    • Export Citation
  • Casati, B., and Coauthors, 2008: Forecast verification: Current status and future directions. Meteor. Appl., 15, 318.

  • Chapman, M., R. Bullock, B. G. Brown, C. A. Davis, K. W. Manning, R. Morss, and A. Takacs, 2004: An object-oriented approach to the verification of quantitative precipitation forecasts: Part II—Examples. Preprints, 17th Conf. on Probability and Statistics in the Atmospheric Sciences/20th Conf. on Weather Analysis and Forecasting/16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., J12.5. [Available online at http://ams.confex.com/ams/pdfpapers/70881.pdf.]

    • Search Google Scholar
    • Export Citation
  • Davis, C., B. Brown, and R. Bullock, 2006a: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev., 134, 17721784.

    • Search Google Scholar
    • Export Citation
  • Davis, C., B. Brown, and R. Bullock, 2006b: Object-based verification of precipitation forecasts. Part II: Application to convective rain systems. Mon. Wea. Rev., 134, 17851795.

    • Search Google Scholar
    • Export Citation
  • Davis, C., B. G. Brown, R. Bullock, and J. Halley Gotway, 2009: The Method for Object-based Diagnostic Evaluation (MODE) applied to numerical forecasts from the 2005 NSSL/SPC Spring program. Wea. Forecasting, 24, 12521267.

    • Search Google Scholar
    • Export Citation
  • Donaldson, R. J., R. M. Dyer, and R. M. Kraus, 1975: An objective evaluator of techniques for predicting severe weather events. Preprints, Ninth Conf. on Severe Local Storms, Norman, OK, Amer. Meteor. Soc., 321–326.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., 2009: Neighborhood verification: A strategy for rewarding close forecasts. Wea. Forecasting, 24, 14981510.

  • Ebert, E. E., and J. L. McBride, 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239, 179202.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., and W. A. Gallus Jr., 2009: Toward better understanding of the contiguous rain area (CRA) method for spatial forecast verification. Wea. Forecasting, 24, 14011415.

    • Search Google Scholar
    • Export Citation
  • Gallus, W. A., 2010: Application of object-based verification techniques to ensemble precipitation forecasts. Wea. Forecasting, 25, 144158.

    • Search Google Scholar
    • Export Citation
  • Gandin, L. S., and A. H. Murphy, 1992: Equitable skill scores for categorical forecasts. Mon. Wea. Rev., 120, 361370.

  • Gilleland, E., D. Ahijevych, B. G. Brown, B. Casati, and E. E. Ebert, 2009: Intercomparison of spatial forecast verification methods. Wea. Forecasting, 24, 14161430.

    • Search Google Scholar
    • Export Citation
  • Kuo, Y.-H., and Y.-R. Guo, 1989: Dynamic initialization using observations from a hypothetical network of profilers. Mon. Wea. Rev., 117, 19751998.

    • Search Google Scholar
    • Export Citation
  • Kuo, Y.-H., E. G. Donall, and M. A. Shapiro, 1987: Feasibility of short-range numerical weather prediction using observations from a network of wind profilers. Mon. Wea. Rev., 115, 24022427.

    • Search Google Scholar
    • Export Citation
  • Mass, C. F., D. Ovens, K. Westrick, and B. A. Colle, 2002: Does increasing horizontal resolution produce more skillful forecasts? Bull. Amer. Meteor. Soc., 83, 407430.

    • Search Google Scholar
    • Export Citation
  • Moninger, W. R., S. G. Benjamin, B. D. Jamison, T. W. Schlatter, T. L. Smith, and E. J. Szoke, 2010: Evaluation of regional aircraft observations using TAMDAR. Wea. Forecasting, 25, 627645.

    • Search Google Scholar
    • Export Citation
  • National Research Council, 2009: Observing Weather and Climate from the Ground Up: A Nationwide Network of Networks. The National Academies Press, 250 pp. [Available online at http://www.nap.edu/catalog/12540.html.]

    • Search Google Scholar
    • Export Citation
  • Otkin, J. A., D. C. Hartung, D. D. Turner, R. A. Petersen, W. F. Feltz, and E. Janzon, 2011: Assimilation of surface-based boundary layer profiler observations during a cool-season weather event using an observing system simulation experiment. Part I: Analysis impact. Mon. Wea. Rev., 139, 2309–2326.

    • Search Google Scholar
    • Export Citation
  • Rostkier-Edelstein, D., and J. P. Hacker, 2010: The roles of surface-observation ensemble assimilation and model complexity for nowcasting of PBL profiles: A factor separation analysis. Wea. Forecasting, 25, 16701690.

    • Search Google Scholar
    • Export Citation
  • Schaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting, 5, 570575.

  • Skamarock, W. C., J. B. Klemp, J. Dudhia, D. O. Gill, D. M. Barker, W. Wang, and J. G. Powers, 2005: A description of the Advanced Research WRF version 2. NCAR Tech. Note/TN-468+STR, 88 pp.

    • Search Google Scholar
    • Export Citation
  • Smith, T. L., S. G. Benjamin, S. I. Gutman, and S. Sahm, 2007: Short-range forecast impact from assimilation of GPS-IPW observations into the Rapid Update Cycle. Mon. Wea. Rev., 135, 29142930.

    • Search Google Scholar
    • Export Citation
  • Stein, U., and P. Alpert, 1993: Factor separation in numerical simulations. J. Atmos. Sci., 50, 21072115.

  • Weisman, M. L., C. Davis, W. Wang, K. W. Manning, and J. B. Klemp, 2008: Experiences with 0–36-h explicit convective forecasts with the WRF-ARW model. Wea. Forecasting, 23, 407437.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences: An Introduction. Academic Press, 467 pp.

  • World Meteorological Organization, 2003: Aircraft Meteorological Data Relay (AMDAR) reference manual. WMO Publication 958, World Meteorological Organization, Geneva, Switzerland, 80 pp.

    • Search Google Scholar
    • Export Citation
Save
  • Ahjevych, D., E. Gilleland, B. Brown, and E. Ebert, 2009: Application of spatial verification methods to idealized and NWP gridded precipitation forecasts. Wea. Forecasting, 24, 14851497.

    • Search Google Scholar
    • Export Citation
  • AIRDAT, 2004: TAMDAR sensor and system overview: Airborne real time atmospheric data collection and reporting system. AIRDAT, LLC, Morrisville, NC, 10 pp. [Available online at http://www.airdat.com/tamdar/tech_docs/TAMDAR%20Overview%20101904.pdf.]

    • Search Google Scholar
    • Export Citation
  • Anderson, J., T. Hoar, K. Raeder, H. Liu, N. Collins, R. Torn, and A. Avellano, 2009: The Data Assimilation Research Testbed: A community facility. Bull. Amer. Meteor. Soc., 90, 12831296.

    • Search Google Scholar
    • Export Citation
  • Baldwin, M. E., and J. S. Kain, 2006: Sensitivity of several performance measures to displacement error, bias, and event frequency. Wea. Forecasting, 21, 636648.

    • Search Google Scholar
    • Export Citation
  • Benjamin, S. G., K. A. Brewster, R. Brümmer, B. F. Jewett, T. W. Schlatter, T. L. Smith, and P. A. Stamus, 1991: An isentropic 3-hourly data assimilation system using ACARS aircraft observations. Mon. Wea. Rev., 119, 888906.

    • Search Google Scholar
    • Export Citation
  • Benjamin, S. G., B. D. Jamison, W. R. Moninger, S. R. Sahm, B. E. Schwartz, and T. W. Schlatter, 2010: Relative short-range forecast impact from aircraft, profiler, radiosonde, VAD, GPS-PW, METAR, and mesonet observations via the RUC hourly assimilation cycle. Mon. Wea. Rev., 138, 13191343.

    • Search Google Scholar
    • Export Citation
  • Casati, B., and Coauthors, 2008: Forecast verification: Current status and future directions. Meteor. Appl., 15, 318.

  • Chapman, M., R. Bullock, B. G. Brown, C. A. Davis, K. W. Manning, R. Morss, and A. Takacs, 2004: An object-oriented approach to the verification of quantitative precipitation forecasts: Part II—Examples. Preprints, 17th Conf. on Probability and Statistics in the Atmospheric Sciences/20th Conf. on Weather Analysis and Forecasting/16th Conf. on Numerical Weather Prediction, Seattle, WA, Amer. Meteor. Soc., J12.5. [Available online at http://ams.confex.com/ams/pdfpapers/70881.pdf.]

    • Search Google Scholar
    • Export Citation
  • Davis, C., B. Brown, and R. Bullock, 2006a: Object-based verification of precipitation forecasts. Part I: Methodology and application to mesoscale rain areas. Mon. Wea. Rev., 134, 17721784.

    • Search Google Scholar
    • Export Citation
  • Davis, C., B. Brown, and R. Bullock, 2006b: Object-based verification of precipitation forecasts. Part II: Application to convective rain systems. Mon. Wea. Rev., 134, 17851795.

    • Search Google Scholar
    • Export Citation
  • Davis, C., B. G. Brown, R. Bullock, and J. Halley Gotway, 2009: The Method for Object-based Diagnostic Evaluation (MODE) applied to numerical forecasts from the 2005 NSSL/SPC Spring program. Wea. Forecasting, 24, 12521267.

    • Search Google Scholar
    • Export Citation
  • Donaldson, R. J., R. M. Dyer, and R. M. Kraus, 1975: An objective evaluator of techniques for predicting severe weather events. Preprints, Ninth Conf. on Severe Local Storms, Norman, OK, Amer. Meteor. Soc., 321–326.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., 2009: Neighborhood verification: A strategy for rewarding close forecasts. Wea. Forecasting, 24, 14981510.

  • Ebert, E. E., and J. L. McBride, 2000: Verification of precipitation in weather systems: Determination of systematic errors. J. Hydrol., 239, 179202.

    • Search Google Scholar
    • Export Citation
  • Ebert, E. E., and W. A. Gallus Jr., 2009: Toward better understanding of the contiguous rain area (CRA) method for spatial forecast verification. Wea. Forecasting, 24, 14011415.

    • Search Google Scholar
    • Export Citation
  • Gallus, W. A., 2010: Application of object-based verification techniques to ensemble precipitation forecasts. Wea. Forecasting, 25, 144158.

    • Search Google Scholar
    • Export Citation
  • Gandin, L. S., and A. H. Murphy, 1992: Equitable skill scores for categorical forecasts. Mon. Wea. Rev., 120, 361370.

  • Gilleland, E., D. Ahijevych, B. G. Brown, B. Casati, and E. E. Ebert, 2009: Intercomparison of spatial forecast verification methods. Wea. Forecasting, 24, 14161430.

    • Search Google Scholar
    • Export Citation
  • Kuo, Y.-H., and Y.-R. Guo, 1989: Dynamic initialization using observations from a hypothetical network of profilers. Mon. Wea. Rev., 117, 19751998.

    • Search Google Scholar
    • Export Citation
  • Kuo, Y.-H., E. G. Donall, and M. A. Shapiro, 1987: Feasibility of short-range numerical weather prediction using observations from a network of wind profilers. Mon. Wea. Rev., 115, 24022427.

    • Search Google Scholar
    • Export Citation
  • Mass, C. F., D. Ovens, K. Westrick, and B. A. Colle, 2002: Does increasing horizontal resolution produce more skillful forecasts? Bull. Amer. Meteor. Soc., 83, 407430.

    • Search Google Scholar
    • Export Citation
  • Moninger, W. R., S. G. Benjamin, B. D. Jamison, T. W. Schlatter, T. L. Smith, and E. J. Szoke, 2010: Evaluation of regional aircraft observations using TAMDAR. Wea. Forecasting, 25, 627645.

    • Search Google Scholar
    • Export Citation
  • National Research Council, 2009: Observing Weather and Climate from the Ground Up: A Nationwide Network of Networks. The National Academies Press, 250 pp. [Available online at http://www.nap.edu/catalog/12540.html.]

    • Search Google Scholar
    • Export Citation
  • Otkin, J. A., D. C. Hartung, D. D. Turner, R. A. Petersen, W. F. Feltz, and E. Janzon, 2011: Assimilation of surface-based boundary layer profiler observations during a cool-season weather event using an observing system simulation experiment. Part I: Analysis impact. Mon. Wea. Rev., 139, 2309–2326.

    • Search Google Scholar
    • Export Citation
  • Rostkier-Edelstein, D., and J. P. Hacker, 2010: The roles of surface-observation ensemble assimilation and model complexity for nowcasting of PBL profiles: A factor separation analysis. Wea. Forecasting, 25, 16701690.

    • Search Google Scholar
    • Export Citation
  • Schaefer, J. T., 1990: The critical success index as an indicator of warning skill. Wea. Forecasting, 5, 570575.

  • Skamarock, W. C., J. B. Klemp, J. Dudhia, D. O. Gill, D. M. Barker, W. Wang, and J. G. Powers, 2005: A description of the Advanced Research WRF version 2. NCAR Tech. Note/TN-468+STR, 88 pp.

    • Search Google Scholar
    • Export Citation
  • Smith, T. L., S. G. Benjamin, S. I. Gutman, and S. Sahm, 2007: Short-range forecast impact from assimilation of GPS-IPW observations into the Rapid Update Cycle. Mon. Wea. Rev., 135, 29142930.

    • Search Google Scholar
    • Export Citation
  • Stein, U., and P. Alpert, 1993: Factor separation in numerical simulations. J. Atmos. Sci., 50, 21072115.

  • Weisman, M. L., C. Davis, W. Wang, K. W. Manning, and J. B. Klemp, 2008: Experiences with 0–36-h explicit convective forecasts with the WRF-ARW model. Wea. Forecasting, 23, 407437.

    • Search Google Scholar
    • Export Citation
  • Wilks, D. S., 1995: Statistical Methods in the Atmospheric Sciences: An Introduction. Academic Press, 467 pp.

  • World Meteorological Organization, 2003: Aircraft Meteorological Data Relay (AMDAR) reference manual. WMO Publication 958, World Meteorological Organization, Geneva, Switzerland, 80 pp.

    • Search Google Scholar
    • Export Citation
  • Fig. 1.

    Conceptual model for the OSSE. Step 1 involved the generation of the initial ensemble that is integrated forward in time 24 h. Conventional observations are assimilated during step 2 with both conventional and ground-based profiler observations assimilated in step 3. Finally, 12-h forecasts are performed during step 4.

  • Fig. 2.

    WRF model domain for all forecast runs (terrain elevation in meters above mean sea level shaded) with the forecast statistical verification subdomain outlined with a solid white line.

  • Fig. 3.

    (a) Simulated 500-hPa geopotential height (shaded every 20 m) and winds (m s−1) valid at 0000 UTC 8 Jan 2008. (b) Simulated 100-m AGL water vapor mixing ratio (g kg−1; shaded), winds (m s−1), and temperature (contoured in black every 2°C) valid at 0000 UTC 8 Jan 2008. (c),(d),(e),(f) As in (a),(b), but valid at 0600 UTC 8 Jan and 1200 UTC 8 Jan 2008, respectively. Dashed white lines denote the location of trough axes.

  • Fig. 4.

    Vertical profiles of (a) root-mean-square error reduction and (b) difference in bias [right side of (a) and (b)] for temperature (K) computed by subtracting the error profile for a given case from the CONV profile shown on the left side of (a) and (b). The profiles were computed using data from the 0-h analysis ensemble mean valid at 0000 UTC 8 Jan in the forecast verification region shown in Fig. 2. (c),(d) As in (a),(b), but for relative humidity (%). (e),(f) As in (a),(b), but for wind vector speed (m s−1). Results are shown for the CONV-DWL (dashed green), CONV-RAM (dashed blue), CONV-RD (dashed red), CONV-MD (blue), CONV-AD (green), and CONV-MAD (red) experiments.

  • Fig. 5.

    As in Fig. 4, but for the 6-h forecast valid at 0600 UTC 8 Jan 2008. All statistics were computed using the forecast ensemble mean.

  • Fig. 6.

    Simulated 6-h accumulated precipitation (mm; shaded), 4-km AGL wind (m s−1), and 850-hPa equivalent potential temperature (contoured in black every 4 K) valid at (a) (left) 0600 UTC 8 Jan 2008 and (b) (right) 1200 UTC 8 Jan 2008 for the truth simulation. (c)–(p) As in (a),(b), but valid for ensemble mean forecasts of (c),(d) CONV; (e),(f) CONV-DWL; (g),(h) CONV-RAM; (i),(j) CONV-RD; (k),(l) CONV-MD; (m),(n) CONV-AD; and (o),(p) CONV-MAD assimilation experiments. Red dashed lines indicate the position of the trough axis at 4 km above ground level.

  • Fig. 7.

    Simulated 850-hPa moisture flux divergence (g kg−1 s−1; contoured every 0.8 × 10−4 g kg−1 s−1) and 850-hPa moisture flux vectors (kg kg−1 m s−1) valid at (a) (left) 0300 UTC 8 Jan 2008 and (b) (right) 0900 UTC 8 Jan 2008 for the truth simulation. (c)–(p) As in (a),(b), but valid for ensemble mean forecasts of (c),(d) CONV; (e),(f) CONV-DWL; (g),(h) CONV-RAM; (i),(j) CONV-RD; (k),(l) CONV-MD; (m),(n) CONV-AD; and (o),(p) CONV-MAD assimilation experiments.

  • Fig. 8.

    (a) Equitable threat scores for 6-h accumulated precipitation (mm) using conventional data only valid at 0600 and 1200 UTC 8 Jan 2008 at the following thresholds: >0.20 mm (medium gray; shaded), >2.4 mm (light gray; shaded), >6.0 mm (black; shaded), >12.0 mm (dark gray; shaded), and >24.0 mm (solid white). (b) Equitable threat score differences from conventional only data for the 0600–1200 UTC 8 Jan 2008 forecast period for each 6-h accumulated precipitation intensity threshold shown in (a). Forecasts from all assimilation experiments shown in Table 1 are displayed.

  • Fig. 9.

    (a) Critical success index for 6-h accumulated precipitation (mm) using conventional data only valid at 0600 and 1200 UTC 8 Jan 2008 at the following thresholds: >0.20 mm (medium gray; shaded), >2.4 mm (light gray; shaded), >6.0 mm (black; shaded), >12.0 mm (dark gray; shaded), and >24.0 mm (solid white). (b) Critical success index differences from conventional only data for the 0600–1200 UTC 8 Jan 2008 forecast period for each 6-h accumulated precipitation intensity threshold shown in (a). Forecasts from all assimilation experiments shown in Table 1 are displayed.

  • Fig. 10.

    MODE statistical analysis of 6-h accumulated precipitation (ACPC) (mm; shaded) objects greater than 6.25 mm ACPC valid at (left) 0600 UTC 8 Jan and (right) 1200 UTC 8 Jan 2008. (a),(b) Identify the “truth” simulation 6-h ACPC objects (contoured yellow) that are compared to the ensemble-mean forecast ACPC objects (contoured red) from the (c),(d) CONV; (e),(f) CONV-DWL; (g),(h) CONV-RAM; (i),(j) CONV-RD; (k),(l) CONV-MD; (m),(n) CONV-AD; and (o),(p) CONV-MAD assimilation experiments. Truth objects introduced in (a),(b) are also shown at their respective times in each of the assimilation forecasts to assist in interpretation. Light gray shaded region denotes the domain used for the ACPC forecast analysis.

All Time Past Year Past 30 Days
Abstract Views 0 0 0
Full Text Views 342 69 5
PDF Downloads 253 43 6