1. Introduction
Some of the recent activities at the Cooperative Institute for Research in the Atmosphere (CIRA) have been related to the development of synthetic satellite imagery (Greenwald et al. 2002; Grasso and Greenwald 2004; Grasso et al. 2008). The motivation for this activity was to evaluate the performance of a numerical weather prediction model using synthetic satellite imagery. Synthetic imagery was produced from the European Center for Medium-Range Weather Forecasts (ECMWF) operational model, and one of its original applications was undertaken by Morcrette (1991). This imagery was subsequently compared to observed Meteosat satellite data. One of the findings in the study by Morcrette (1991) was that simulated brightness temperatures of some clouds were too warm; furthermore, the ECMWF model produced values of cloud coverage and cloud liquid water that were too small. Their results suggested that improvements were needed in the prediction of both cloud coverage and cloud liquid water. Similarly, synthetic satellite imagery was employed by Chaboureau and Pinty (2006) to evaluate and improve the performance of a cirrus parameterization scheme in their numerical model. Greenwald et al. (2002) described the development of an observational operator designed to produce synthetic radiances of the Geostationary Operational Environmental Satellite-9 (GOES-9) imager. In their study, synthetic imagery was produced from the Regional Atmospheric Modeling System (RAMS; Cotton et al. 2003). In their version of RAMS, cloud water only employs one-moment prediction: mass-mixing ratio; cloud droplet number concentrations are specified. Their model results were used to simulate an event characterized by a low-level stratus cloud layer composed entirely of liquid water. Results indicated a reasonable agreement between synthetic and observed GOES-9 imagery. The observational operator was then further developed (Grasso and Greenwald 2004) to calculate optical properties of several new hydrometeor species: pristine ice, snow, aggregates, graupel, hail, and rainwater. All hydrometeor types were computed using two-moment microphysics except cloud liquid. Grasso and Greenwald (2004) reported on the performance of the expanded system for an idealized thunderstorm simulation. The same observational operator is exploited in the current study.
The main goal of the present study is to assess the evaluation of the performance of a numerical model through the use of synthetic imagery. For this purpose, synthetic GOES-10 imagery at 10.7 μm is produced using output from the Advanced Research Weather Research and Forecasting (ARW-WRF; Skamarock et al. 2007; Wicker and Skamarock 2002; Michalakes et al. 1998) version 3.0 community numerical model. The ARW-WRF is a relatively new model and the use of synthetic imagery may provide a unique way to indirectly evaluate the performance of several different microphysical algorithms. Recently, Otkin et al. (2009) successfully produced synthetic imagery of the Spinning Enhanced Visible and Infrared Imager (SEVIRI) that is part of the Meteosat Second Generation (MSG) satellite. In their study, imagery was produced from output of version 2.2 of the WRF model that covered a relatively large domain.
Traditionally, the performance of a numerical model has been evaluated by comparing simulated kinematic and/or thermodynamic fields to observed fields. For example, some of the fields may be surface winds, temperature, dewpoint temperature, pressure, and precipitation amounts. Since operational satellites detect clouds, synthetic satellite imagery of model output can be compared with observed satellite imagery. In this way, one additional metric will be available to evaluate the performance of the numerical model. Since simulated clouds are influenced by the choice of the microphysical scheme in the ARW-WRF, the comparison of brightness temperatures of simulated clouds using various microphysical schemes may yield additional and new information with respect to the performance of those schemes. For example, Grasso et al. (2010) used synthetic satellite imagery to identify coding error in the microphysical routines in the RAMS model. Correctly predicting clouds in the model can be very important because clouds affect the incoming and outgoing radiation, which can, for example, alter the development of convection. In the present study, the simulation of a significant precipitation event associated with an “atmospheric river” affecting the California coast late in December 2005 was performed by using five different microphysics schemes: Lin, WRF single-moment 6 class (WSM6), Thompson, Schultz, and Morrison.
This manuscript is organized as follows. The observed atmospheric river event is described in section 2. The data and methodologies employed are provided in section 3. The core results are discussed in section 4 and summarized in section 5.
2. 30–31 December 2005 atmospheric river event
During the winter season, notable precipitation events in California are usually caused by land-falling “atmospheric rivers,” which are elongated regions of enhanced water vapor flux (e.g., Ralph et al. 2004, 2005; Bao et al. 2006) that typically reside in the warm sector of oceanic extratropical cyclones. Atmospheric rivers are readily identifiable by Special Sensor Microwave Imager (SSM/I) (Hollinger et al. 1990) polar-orbiting satellite imagery of integrated water vapor (IWV). For example, Fig. 1 shows a long, narrow plume of enhanced IWV extending from the tropical western Pacific to the west coast of northern California. The landfall of atmospheric rivers in western North America, such as is shown in Fig. 1, are associated with strong low-level, moist southwesterly flow characterized by weak static stability impinging on the steep coastal topography (e.g., Ralph et al. 2004, 2005; Neiman et al. 2008a,b). Sustained precipitation events are further modified and enhanced significantly by the complex terrain. These events have a major impact on West Coast hydrology (Ralph et al. 2006; Neiman et al. 2008a,b), including the generation of flash flooding and debris flows, given the terrain steepness and soil characteristics.
SSM/I composite images of IWV (cm) between 0000 and 1200 UTC 30 Dec 2005.
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
Numerical simulations of the atmospheric river event that occurred on 30–31 December 2005 were performed when rain amounts exceeding 200 mm fell in the mountainous terrain at some locations in California. Figure 1 shows a narrow, elongated plume of enhanced IWV extending northeastward from the tropical western Pacific Ocean to California at 1200 UTC 30 December during the early stages of this event. The resulting 24-h storm-total precipitation analysis for two 24-h periods between 1200 UTC 29 December and 1200 UTC 31 December 2005 (Figs. 2a and 2b, respectively) show the largest accumulations in California’s northern coastal mountains and in the northern Sierra Nevada. Detailed mesoscale analyses of this event are discussed in Reeves et al. (2008), Jankov et al. (2009), and Smith et al. (2010).
24-h quantitative precipitation estimates from the National Oceanic and Atmospheric Administration (NOAA)’s Advanced Hydrological Prediction Service (http://water.weather.gov) ending at (a) 1200 UTC 30 Dec 2005 and (b) 1200 UTC 31 Dec 2005.
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
Synoptic-scale conditions for this event were assessed using the coarse resolution (~2.5° latitude × ~2.5° longitude) global gridded dataset from the National Centers for Environmental Prediction–National Center for Atmospheric Research (NCEP–NCAR) Reanalysis Project (NNRP; Kalnay et al. 1996). Daily mean and anomaly1 fields for the onset of the event on 30 December 2005 are shown in Fig. 3. The 500-hPa geopotential height analysis (Fig. 3a) portrays a long fetch of strong west–southwesterly flow situated between a deep cyclone over the Gulf of Alaska and ridging over Hawaii. This fetch enters the inner nest of the ARW-WRF domain and terminates over the western United States. The corresponding height anomaly field (Fig. 3e) reveals a prominent couplet, with a >330-m negative anomaly over the Gulf of Alaska and a ~100-m positive anomaly across the subtropical eastern Pacific. The 925-hPa height fields (Figs. 3b,f) mirror the conditions observed aloft, except the ridge anomaly is weaker. An equivalent potential temperature (θe) analysis at 925 hPa (Fig. 3c) shows polar cold frontal baroclinicity extending northeastward from north of Hawaii to the Pacific Northwest. South of the front, a plume of high-θe air exits the tropics and is directed toward the WRF inner nest and northern California. This plume is characterized by positive anomalies in excess of 20 K (Fig. 3g). The θe plume coincides approximately with the axis of maximum vertically integrated horizontal water vapor transport2 (IVT; see Figs. 3d,h). This corridor of enhanced IVT defines the position of the atmospheric river. The maximum mean value of IVT exceeds 1200 kg s−1 m−1, which is much stronger than the mean wintertime value of ~600 (~400) kg s−1 m−1 for 29 (35) atmospheric rivers making landfall in the Pacific Northwest (California) during eight water years between 1997–2005 (Neiman et al. 2008b).
Composite (a)–(d) mean and (e)–(h) anomaly analyses derived from the NNRP reanalysis dataset for 30 Dec 2005. The fields are as follows: (a),(e) 500-hPa geopotential height (m); (b),(f) 925-hPa geopotential height (m); (c),(g) 925-hPa equivalent potential temperature (θe, K); and (d),(h) vertically integrated horizontal water vapor flux (IVT, kg s−1 m−1). The dotted inset boxes in each panel show the outer and inner ARW-WRF domains.
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
3. Data and methodology
ARW-WRF simulations of the 30–31 December 2005 event were carried out by using five different microphysical algorithms: Lin, WSM6, Thompson, Schultz, and Morrison. All five schemes partition condensed water into cloud liquid, cloud ice, rain, snow, and graupel. The Lin scheme is based on Lin et al. (1983), Chen and Sun (2002), and Rutledge and Hobbs (1984), with modifications for saturation adjustment and ice sedimentation following Tao et al. (1989). The WSM6 scheme also followed Tao et al. (1989), with a different accretion calculation (Hong and Lim 2006). Time splitting is applied to the freezing and melting processes to increase accuracy in the vertical heating profile. The saturation adjustment follows Dudhia (1989) and Hong et al. (1998) in separately treating ice and water saturation processes. The Thompson et al. (2004) scheme carries an additional prognostic variable for the number concentration of cloud ice. Primary ice nucleation is calculated according to Cooper (1986) and the autoconversion as in Walko et al. (1995). A generalized gamma function represents the graupel category instead of the exponential representation used in Lin and WSM6. The Schultz (1995) scheme has been modified to use the Asai (1965) saturation adjustment method to slow the melting rate of snow in air slightly warmer than freezing and to allow for the formation of cloud liquid water in unsaturated grid volumes with lapse rates approaching convective instability. Finally, a version of Morrison’s double-moment scheme (Morrison and Pinto 2005, 2006) recently implemented into the ARW-WRF model was used. In this version, all hydrometeors are double moment (mass-mixing ratio and number concentration predicted) except for cloud water. For all simulations, precipitations were resolved explicitly. Each of the five microphysics configurations used the nonlocal mixing Yonsei University (YSU) PBL scheme as an improved version of the Medium-Range Forecast (MRF) Model PBL scheme (Troen and Mahrt 1986). For longwave radiation the Rapid Radiative Transfer Model scheme was used (Mlawer et al. 1997), while for the shortwave radiation the Dudhia (1989) scheme was employed. Land surface processes were resolved by using the Noah Land Surface Model scheme with soil temperature and moisture at four layers (Chen and Dudhia 2001), fractional snow cover, and frozen soil physics. In the ARW-WRF model, the three-dimensional domain employs an Arakawa C staggered grid. In terms of vertical resolution, all simulations were performed with 51 vertical layers.
All model runs were initialized at 1200 UTC 30 December 2005 and North American Mesoscale (NAM) model analysis and forecasts were used for initial and boundary conditions, respectively. The integrations were performed over a 24-h period and contained two domains, one of which was nested (the inset domains in Fig. 3 show these domains). In the present study, the one-way nesting technique was employed. Grid spacings in these domains were 20 and 4 km, respectively. The grid spacing of 4 km for the inner grid was chosen to match the approximate footprint of GOES-10. For the statistical analysis, a 15-min model output was used. The output from ARW-WRF was then utilized as an input to the observational operator, and this operator was used to create the synthetic satellite imagery. The three main components of the operator are a gas extinction model, hydrometeor optical property models, and radiative transfer models. Gas extinction was computed using optical transmittance (OPTRAN) code (McMillin et al. 1995). Modified anomalous diffraction theory (Mitchell 2000) was used to compute hydrometeor optical properties. The gaseous extinction and hydrometeor properties were fed into a radiative transfer model (Deeter and Evans 1998), which computes synthetic brightness temperatures. The accuracy of brightness temperature computed by the radiative transfer model was shown to be 1.5–2 K. In a study by Grasso et al. (2010), a 2% cold bias in synthetic satellite imagery was found when compared to GOES-12 observational data. Four out of five (all except for the double-moment Morrison scheme) microphysical schemes used in this study only predict the mass-mixing ratio for each hydrometeor. The observational operator requires not only a mass-mixing ratio, but also number concentration. As a result, for these four, physics particle number concentrations were specified. The issue of specifying a number concentration for a single-moment scheme was first addressed as a part of a separate study. The observational operator was originally developed for the RAMS model; as a result, RAMS was used to estimate values of number concentrations to supplement the ARW-WRF microphysics. This was done by running RAMS simulations, taking vertical cross sections through different habit types, and extracting max values of number concentrations. These values were then specified in the observational operator for single-moment ARW-WRF microphysics. A series of experiments were performed to compare the RAMS versus ARW-WRF synthetic imagery as well as the sensitivity to changes in number concentration specifications. It was found that the difference in simulated brightness temperature between the two models was approximately two degrees. With regard to sensitivity to prescribed number concentrations, similar results were obtained when the value of number concentrations were varied by an order of magnitude.
Brightness temperatures for simulations using various microphysics algorithms were calculated and statistically compared with observations using histograms and cumulative frequency distribution (CFD). In addition, various objective skill measures were computed, such as false alarm ratio (FAR), probability of detection (POD), threat score (TS), and bias (ratio between numbers of forecasted and observed points that exceed the threshold).
4. Results
a. Comparison of standard simulated fields to available analyses
Traditionally, for an evaluation of a numerical model performance, simulated fields such as 500-hPa geopotential heights, surface wind, surface temperature, and accumulated precipitation are compared to available observations. In the present study we compare simulated 500-hPa geopotential heights (Fig. 4a) and 925-hPa temperatures (Fig. 5a) to NCEP’s North American Regional Reanalysis (NARR) over the outer ARW-WRF nest (Figs. 4b and 5b). For this purpose, the model simulation using the WSM6 microphysics was chosen. It is noteworthy that no major differences in these fields among model runs using various microphysics schemes were noted (not shown). Comparing to both NARR (Fig. 4b) and available global NNRP analyses at 500 hPa (Figs. 4b and 3a, respectively), the model captured the synoptic-scale trough over the eastern Pacific, just west of California. Also, simulated ridging over central and Southern California agreed well with both available analyses.
500-hPa geopotential height (m): (a) 12-h model simulation over the outer domain and (b) NARR mean field valid at 0000 UTC 31 Dec 2005.
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
As in Fig. 4, but for 925-hPa temperature (°C) valid at 0000 UTC 31 Dec 2005.
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
In terms of 925-hPa temperature (Fig. 5a), the model properly depicted a strong temperature gradient over the northwestern quadrant of the ARW-WRF outer nest associated with the cold polar front (vicinity of the isotherms labeled with 13 and 14). Location of the warm sector and corresponding values of the temperature within the warm sector compared favorably with observations. The IWV simulated by a model run using the WSM6 microphysics (Fig. 6) shows a land-falling atmospheric river affecting the northern half of California with the highest value of IWV centered over the bay area. The model runs using different microphysics schemes resulted in almost identical simulations of the IWV (not shown). The general location of the atmospheric river landfall simulated by the models agreed well with available reanalysis.
Simulated IWV (mm) valid at 0000 UTC 31 Dec 2005.
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
Twenty-four-hour precipitation accumulations valid at 1200 UTC 31 December 2005 from the model runs using various microphysics are presented in Fig. 7. For all model simulations, a general distribution of the heaviest amounts being simulated in California’s northern coastal mountains and northern Sierra Nevada ranges was consistent with the daily observed accumulations presented in Fig. 2b. By comparing the performance of various microphysical schemes relative to each other, it can be seen that the model run using Lin microphysics resulted in a much larger areal coverage and heavier amounts compared to all other simulations and observations (Fig. 2b). The other simulations generated very similar precipitation accumulations in terms of both areal coverage and the amounts.
The 24-h precipitation accumulations (mm) over the inner nest simulated by model runs using Lin, WSM6, Thompson, Schultz, and two-moment Morrison microphysics valid at 1200 UTC 31 Dec 2005.
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
Figure 8 shows calculated equitable threat scores (ETSs) for the five model runs using various microphysics for both 12-h (Fig. 8a) and 24-h (Fig. 8b) accumulation periods. In accordance with the subjective evaluation, the model run using Lin microphysics resulted in lower ETSs compared to all other model runs. For example, 12-h forecasts were characterized with almost no skill (ETS values close to 0) for all precipitation thresholds. For 24-h forecast and lower-precipitation thresholds, the model run using Lin microphysics was characterized with ETSs of ~0.4, while for higher precipitation thresholds, ETSs decreased to ~0.2. All other model runs were generally comparable at both times (ETS values on the order of ~0.4–0.5 for 12-h forecasts and ~0.6–0.7 for 24-h forecasts), except the model run using Schultz microphysics had notably lower skill for lighter thresholds. The root-mean-square error (RMSE) (Fig. 8c) calculated for all model runs and for the two evaluation periods also pointed toward precipitation overestimation for the model run using Lin microphysics. The RMSE values for the rest of the model runs were similar except that they were slightly higher for the model run using the WSM6 microphysics.
ETS calculated for the five model runs using different microphysics for (a) 12-h and (b) 24-h precipitation accumulations. (c) Corresponding RMSE values.
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
b. Comparison of simulated and observed GOES-10 10.7-μm brightness temperatures
Even though precipitation fields simulated by various microphysics were generally similar in terms of areal coverage and distribution—with the exception of the Lin scheme (Fig. 7)—simulated brightness temperatures showed different results. Figure 9 shows observed brightness temperature and 12-h forecast brightness temperatures simulated by the five different model configurations over the ARW-WRF nested domain, valid at 0000 UTC 31 December 2005. The model run using the Lin microphysics resulted in low values of simulated brightness temperature covering most of the domain. This solution was dominated by high clouds covering a majority of the integration domain (Fig. 9b). The remaining simulations resulted in more realistic cloud-top large-scale patterns and their corresponding brightness temperatures. For example, the cloud-top pattern was better replicated by the model runs using the WSM6 (Fig. 9c) and Morrison microphysics (Fig. 9f). In these two simulations, clouds were characterized by a more “banded” appearance compared to the more “smooth” appearance associated with the simulations from the model runs using the Thomson (Fig. 9d) and Schultz (Fig. 9e) microphysics. The Thompson and Shultz microphysics were also somewhat warmer than observations at this time.
Brightness temperatures (°C) at 0000 UTC 31 Dec 2005: (a) observations by the NOAA GOES-10 satellite, and simulation by the ARW-WRF model using (b) Lin, (c) WSM6, (d) Thompson, (e) Schultz, and (f) Morrison microphysical algorithms.
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
In addition, at this time all model solutions, except the one using the Lin microphysics, were characterized with a warm bias in simulated brightness temperatures over the southern portion of the domain. Therefore, these model configurations simulated a clear sky in cloudy areas.
Similar analyses were performed for 24-h simulations valid at 1200 UTC 31 December 2005. The corresponding results are presented in Fig. 10. It can be seen that the model simulation using the Lin microphysics once again resulted in a solution characterized by lower-than-observed brightness temperatures over approximately two-thirds of the integration domain (Fig. 10b). The model runs using WSM6 (Fig. 10c) and Morrison (Fig. 10f) microphysics resulted in better simulations of general cloud coverage pattern compared to observations. The model simulation using Schultz microphysics (Fig. 10e) was consistently characterized by a smooth cloud appearance. This behavior may be at least partially explained by the water substance partitioning experiment discussed in Jankov et al. (2009). In that study, four microphysical schemes (Lin, WSM6, Thompson, and Schultz) were examined. The results indicated that the Schultz microphysics was characterized by notably more cloud ice compared to all other examined microphysics. Also, the Schultz and Thompson microphysics were characterized by much more snow and much less graupel compared to the other two schemes. The WSM6 microphysics was characterized with cloud water dominating over cloud ice and much larger rain and graupel mixing ratios compared to the snow mixing ratio. These findings from the study performed by Jankov et al. (2009) are illustrated in Fig. 11. Figures 11a–d show the vertical distribution of hydrometeors at California’s coastal location of Cazadero (CZD) for the simulations of 30–31 December 2005 using four different microphysics. Figure 11e illustrates the water substance partition over California’s American River basin for the four different microphysics schemes averaged over five atmospheric river events.
As in Fig. 9, but for 1200 UTC 31 Dec 2005.
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
Snow (color shaded, see color bar), rain (white contours), and graupel (black contours) mixing ratios (kg kg−1), and 0°C temperature (red) and wet-bulb temperature (green) lines for the 30 Dec 2005 case at CZD for model runs using (a) Lin, (b) WSM6, (c) Thompson, and (d) Schultz microphysics. Melting-level heights (represented by *) from available OAK soundings. Both shading and contours are scaled by a factor of 10−4. (e) Cloud water, cloud ice, rain, snow, and graupel mixing ratios (×10−3 kg kg−1) averaged over five cases and over the American River basin for four different microphysics schemes.
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
The area over the southern part of the domain characterized by simulated brightness temperatures being higher than observed, present in most of the 12-h simulations, was not as pronounced at this time.
One convenient way to evaluate the results was to create the probability of occurrence histograms associated with each model configuration and the corresponding observations. Figure 12 illustrates the histogram for simulations valid at 0000 UTC 31 December 2005. At 0000 UTC 31 December 2005, observations indicated two maxima in probability of occurrence (black line in Fig. 11). The primary maximum was centered on ~225 K with the probability of occurrence of ~8% and the secondary centered on ~280 K with the corresponding probability of occurrence being ~3%. For the Lin microphysics model run, the primary maximum was centered at ~215 K and the corresponding probability of occurrence largely overestimated (~24%). By contrast, this solution resulted in very few realizations on the warmer end of the scale. Model runs using the WSM6, Morrison, and Schultz schemes resulted in a primary maximum correctly centered on ~225 K. However, the WSM6 run resulted in a slight overestimation (~11%) of probability of occurrence while the opposite was true for the solution using the Schultz microphysics. The model run using the Thompson microphysics correctly simulated the probability of occurrence of the primary peak, but the corresponding brightness temperature was ~15 K warmer than observed. The secondary maximum was largely overestimated by all model solutions except the one using the Lin scheme. The overestimation was the largest and the smallest for the model runs using the Schultz and Morrison schemes, respectively. Probabilities of occurrence for the range of brightness temperatures between the two maxima were slightly underestimated by all schemes. Once again, the model run using Lin microphysics was the outlier. The analysis valid at 0000 UTC 31 December 2005 indicates that some of the microphysics slightly overestimated or underestimated presence of high clouds (WSM6 and Schultz) or, in the case of the Thompson scheme, the high clouds were warmer than those observed. Overestimation of high values of brightness temperatures by all microphysics, except Lin, points toward a general underestimation of lower-cloud presence. This agrees with previously discussed findings of subjective synthetic satellite imagery analysis, pointing toward the warm bias over the southern portion of the domain.
Probability of occurrence histograms for observed GOES-10 brightness temperatures (K) and synthetic brightness temperatures (K) simulated by model runs using Lin, WSM6, Thompson, Schultz, and Morrison microphysical algorithms valid at 0000 UTC 31 Dec 2005 (12-h forecasts).
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
A similar analysis was performed for the 24-h simulations of brightness temperatures and corresponding observations (not shown). The results were generally comparable to those of the 12-h simulations. In contrast to observations valid at 0000 UTC 31 December 2005, 12 h later, observations were characterized by only one pronounced maximum centered at ~220 K and its corresponding probability of occurrence of ~9%. The model solution using the Lin scheme had the maximum centered over a lower temperature than observed and with overestimated probability of its occurrence. In terms of the other model configurations, at this forecast time, all of them resulted in a slight underestimation of the maximum probability of occurrence. In addition, the solution using the Thompson microphysics, as for the 12-h forecast, produced a probability of occurrence centered over a somewhat higher value of brightness temperature (~227 K) than observed. Compared to the 12-h simulation, a warm bias associated with most of the solutions was notably reduced. This also agreed with the previously discussed subjective analysis of simulated brightness temperatures.
Figure 13 shows cumulative percentage plots of observed GOES-10 10.7-μm brightness temperatures versus those simulated by various microphysical schemes valid at 0000 UTC 31 December 2005 (12-h forecast). This graph reinforced the observation that the simulation using the Lin microphysics departed the most from observations as well as from other solutions by being too “cold” at both simulation times. Overall, 12-h simulations performed by all other model configurations were characterized by a good agreement with observations for lower values of brightness temperatures and a warm bias presence for higher values of brightness temperatures (Fig. 13). A warm bias in brightness temperature may indicate a tendency of the schemes to produce fewer optically thin clouds and more midlevel clouds and/or “clear sky” than observed. The 24-h cumulative percentage analysis (not shown) generally indicated good agreement between all simulations and observations for the whole range of brightness temperature values except when using the Lin microphysics.
Cumulative percentage plots of observed (GOES-10 10.7 μm) vs brightness temperatures simulated by various microphysical schemes valid at 0000 UTC 31 Dec 2005 (12-h forecasts).
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
In addition, POD, TS, FAR, and bias for the 12- and 24-h forecasts were computed (Figs. 14 and 15). It can be seen that the model run using Lin microphysics was characterized with the lowest skill at both times. The Lin microphysics solution resulted in the lowest POD and TS, with occasionally very high FAR and notable bias for the lowest threshold. All other solutions resulted in generally comparable skill measures at both times. For the 12-h forecasts (Fig. 14), the Schultz and Thompson schemes were characterized with higher POD than the other two schemes. At the same time, when compared to the other three schemes, the Schultz scheme resulted in higher FAR for all thresholds except the highest. The WSM6 scheme was characterized with smaller bias than all other solutions. For the 24-h forecasts (Fig. 15) the four schemes had generally very comparable skill scores, except that the Morrison scheme was characterized with notably smaller FAR compared to all other solutions. All results discussed so far were consistent for all 15-min analyzed periods.
Calculation of (a) POD, (b) TS, (c) FAR, and (d) bias for 12-h brightness temperature forecasts, valid at 0000 UTC 30 Dec 2005 for the five different model solutions.
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
As in Fig. 14, but for the 24-h brightness temperature forecasts.
Citation: Journal of Hydrometeorology 12, 4; 10.1175/2010JHM1282.1
5. Summary
The main focus of the present study was to assess the value added by use of synthetic satellite imagery in model evaluation performance in addition to more traditional approaches. For this purpose, an atmospheric river event that generated heavy precipitation in California on 30–31 December 2005 was simulated by using the ARW-WRF numerical model and five different microphysical schemes. The five different microphysics included Lin, WSM6, Thompson, Schultz, and double-moment Morrison schemes. Synthetic imagery was created and the simulations were statistically compared with observations from the 10.7-μm channel of GOES-10 using a histogram-based technique.
The simulated atmospheric river event was characterized by precipitation amounts in excess of 200 mm at both coastal and Sierra Nevada locations of northern California. Model simulations were performed over a 24-h period starting at 1200 UTC 30 December 2005. The model output was produced every 15 min. The integration domain consisted of a large outer domain with 20-km grid spacing and an inner nest with 4-km grid spacing (Fig. 3), which was chosen to match the satellite footprint.
For this event, simulated 500-hPa geopotential heights showed good agreement with both NARR and NNRP analyses. Namely, the strong 500- and 925-hPa southwesterly flow entering the outer ARW-WRF nest, as well as ridging over the central and southern California, were well simulated by the model. Corresponding features in 925-hPa temperature—such as a strong temperature gradient over the northwestern quadrant of the outer nest associated with the cold polar front and an area of warmer air entering the model inner nest from southwest—were depicted by the model as well.
A comparison of 12- (not shown) and 24-h accumulated precipitation amounts simulated by the model using various microphysics showed similar results in terms of areal coverage and distribution, except for the run using Lin microphysics. This run was characterized by both larger areal coverage and heavier amounts compared to all other solutions. This was identified by both subjective and objective evaluations. On the other hand, a comparison of simulated brightness temperatures valid at the same time showed different results. Synthetic imagery for the model run using Lin microphysics notably differed from all other simulations. The solution using Lin microphysics was cooler compared to observations and all other solutions, indicating that the entire domain was covered with high-level clouds. Synthetic imagery from model runs using WSM6, Thompson, Schultz, and Morrison microphysics differed comparably in terms of both cloud-top areal pattern and corresponding brightness temperatures, but were overall comparable to the observations. Statistical analysis of the results indicated that all configurations, except the one using Lin microphysics, simulated high clouds generally well. On the other hand, the model configurations (other than Lin) showed a tendency to underestimate a presence of midlevel clouds and to overestimate the presence of clear sky conditions (better skill score measures for lower-brightness temperature thresholds). This was especially true for the earlier simulation hours (e.g., the 12-h forecasts). At later simulation hours, when the system moved eastward, and the domain of integration was dominated by lower water clouds, the warm bias was reduced. It is noteworthy that the results obtained in the present study are specific for the model setup chosen for the experiment.
Overall, the results in the present study show the potential in using synthetic satellite imagery as a useful model performance evaluation tool. Traditional validation metrics, such as observed precipitation, 500-hPa heights, or surface winds, may suggest that a model simulation agrees quite well with observations, while in reality the model may have generated too many clouds. These clouds could then prohibit surface heating in the model, which in turn could lead to additional errors. Using synthetic satellite data as a validation tool would allow these model errors to be more easily identified. Synthetic satellite imagery can also be very useful for better understanding and improvement of existing microphysical schemes in addition to the identification of errors in cloud generation, which will be a topic of future studies.
Acknowledgments
The authors thank to Daniel Coleman for providing help in obtaining the observational data for the event of interest and Ann Reiser for her help with editing.
REFERENCES
Asai, T., 1965: A numerical study of the air-mass transformation over the Japan Sea in winter. J. Meteor. Soc. Japan, 43, 1–15.
Bao, J.-W., Michelson S. A. , Neiman P. J. , Ralph F. M. , and Wilczak J. M. , 2006: Interpretation of enhanced integrated water vapor bands associated with extratropical cyclones: Their formation and connection to tropical moisture. Mon. Wea. Rev., 134, 1063–1080.
Chaboureau, J.-P., and Pinty J.-P. , 2006: Evaluation of a cirrus parameterization with Meteosat Second Generation observations. Geophys. Res. Lett., 33, L03815, doi:10.1029/2005GL024725.
Chen, F., and Dudhia J. , 2001: Coupling an advanced land surface–hydrology model with the Penn State–NCAR MM5 modeling system. Part I: Model implementation and sensitivity. Mon. Wea. Rev., 129, 569–585.
Chen, S.-H., and Sun W.-Y. , 2002: A one-dimensional time-dependent cloud model. J. Meteor. Soc. Japan, 80, 99–118.
Cooper, W. A., 1986: Ice initiation in natural clouds. Precipitation Enhancement—A Scientific Challenge, Meteor. Monogr., No. 43, Amer. Meteor. Soc., 29–32.
Cotton, W. R., and Coauthors, 2003: RAMS 2001: Current status and future directions. Meteor. Atmos. Phys., 82, 5–29.
Deeter, M., and Evans K. F. , 1998: A hybrid Eddington-single scattering radiative transfer model for computing radiances from thermally emitting atmospheres. J. Quant. Spectrosc. Radiat. Transfer, 60, 635–648.
Dudhia, J., 1989: Numerical study of convection observed during the Winter Monsoon Experiment using a mesoscale two-dimensional model. J. Atmos. Sci., 46, 3077–3107.
Grasso, L. D., and Greenwald T. J. , 2004: Analysis of 10.7-μm brightness temperatures of a simulated thunderstorm with two-moment microphysics. Mon. Wea. Rev., 132, 815–825.
Grasso, L. D., Sengupta M. , Dostalek J. F. , Brummer R. , and DeMaria M. , 2008: Synthetic satellite imagery for current and future environmental satellites. Int. J. Remote Sens., 29, 4373–4384.
Grasso, L. D., Sengupta M. , and DeMaria M. , 2010: Comparison between observed and synthetic 6.5 and 10.7 μm GOES-12 imagery of thunderstorms that occurred on 8 May 2003. Int. J. Remote Sens., 31, 647–663.
Greenwald, T. J., Hertenstein R. , and Vukićević T. , 2002: An all-weather observational operator for radiance data assimilation with mesoscale forecast models. Mon. Wea. Rev., 130, 1882–1897.
Hollinger, J. P., Peirce J. L. , and Poe G. A. , 1990: SSM/I instrument evaluation. IEEE Trans. Geosci. Remote Sens., 28, 781–790.
Hong, S.-Y., and Lim J.-O. J. , 2006: The WRF single-moment 6-class microphysics scheme (WSM6). J. Korean Meteor. Soc., 42, 129–151.
Hong, S.-Y., Juang H.-M. H. , and Zhao Q. , 1998: Implementation of prognostic cloud scheme for a regional spectral model. Mon. Wea. Rev., 126, 2621–2639.
Jankov, I., Bao J.-W. , Neiman P. J. , Schultz P. J. , Yuan H. , and White A. B. , 2009: Evaluation and comparison of microphysical algorithms in ARW-WRF model simulations of atmospheric river events affecting the California coast. J. Hydrometeor., 10, 847–870.
Kalnay, E., and Coauthors, 1996: The NCEP/NCAR 40-Year Reanalysis Project. Bull. Amer. Meteor. Soc., 77, 437–471.
Lin, Y.-L., Farley R. D. , and Orville H. D. , 1983: Bulk parameterization of the snow field in a cloud model. J. Climate Appl. Meteor., 22, 1065–1092.
McMillin, L. M., Crone L. J. , and Kleespies T. J. , 1995: Atmospheric transmittance of an absorbing gas. 5. Improvements to the OPTRAN approach. Appl. Opt., 34, 8396–8399.
Michalakes, J., Dudhia J. , Gill D. , Klemp J. , and Skamarock W. , 1998: Design of a next-generation regional weather research and forecast model. Towards Teracomputing, W. Zwieflhofer and N. Kreitz, Eds., World Scientific, 117–124.
Mitchell, D. L., 2000: Parameterization of the Mie extinction and absorption coefficients for water clouds. J. Atmos. Sci., 57, 1311–1326.
Mlawer, E. J., Taubman S. J. , Brown P. D. , Iacono M. J. , and Clough S. A. , 1997: Radiation transfer for inhomogeneous atmosphere: RRTM, a validated correlated-k model for the longwave. J. Geophys. Res., 102 (D14), 16 663–16 682.
Morcrette J.-J., 1991: Evaluation of model-generated cloudiness: Satellite-observed and model-generated diurnal variability of brightness temperatures. Mon. Wea. Rev., 119, 1205–1224.
Morrison, H., and Pinto J. O. , 2005: Mesoscale modeling of springtime Arctic mixed-phase stratiform clouds using a new two-moment bulk microphysics scheme. J. Atmos. Sci., 62, 3683–3704.
Morrison, H., and Pinto J. O. , 2006: Intercomparison of bulk microphysics scheme in mesoscale simulations of springtime Arctic mixed-phase stratiform clouds. Mon. Wea. Rev., 134, 1880–1900.
Neiman, P. J., Ralph F. M. , Wick G. A. , Kuo Y.-H. , Wee T.-K. , Ma Z. , Taylor G. H. , and Dettinger M. D. , 2008a: Diagnosis of an intense atmospheric river impacting the Pacific Northwest: Storm summary and offshore vertical structure observed with COSMIC satellite retrievals. Mon. Wea. Rev., 136, 4398–4420.
Neiman, P. J., Ralph F. M. , Wick G. A. , Lundquist J. D. , and Dettinger M. D. , 2008b: Meteorological characteristics and overland precipitation impacts of atmospheric rivers affecting the west coast of North America based on eight years of SSM/I satellite observations. J. Hydrometeor., 9, 22–47.
Otkin, J. A., Greenwald T. J. , Sieglaff J. , and Huang H.-L. , 2009: Validation of a large-scale simulated brightness temperature dataset using SEVIRI satellite observations. J. Appl. Meteor. Climatol., 48, 1613–1626.
Ralph, F. M., Neiman P. J. , and Wick G. A. , 2004: Satellite and CALJET aircraft observations of atmospheric rivers over the eastern North Pacific Ocean during the winter of 1997/98. Mon. Wea. Rev., 132, 1721–1745.
Ralph, F. M., Neiman P. J. , and Rotunno R. , 2005: Dropsonde observations in low-level jets over the northeastern Pacific Ocean from CALJET-1998 and PACJET-2001: Mean vertical-profile and atmospheric-river characteristics. Mon. Wea. Rev., 133, 889–910.
Ralph, F. M., Neiman P. J. , Wick G. A. , Gutman S. I. , Dettinger M. D. , Cayan C. R. , and White A. B. , 2006: Flooding on California’s Russian River: The role of atmospheric rivers. Geophys. Res. Lett., 33, L13801, doi:10.1029/2006GL026689.
Reeves, H. D., Lin Y.-L. , and Rotunno R. , 2008: Dynamic forcing and mesoscale variability of heavy precipitation events over the Sierra Nevada mountains. Mon. Wea. Rev., 136, 62–77.
Rutledge, S. A., and Hobbs P. V. , 1984: The mesoscale and microscale structure and organization of clouds and precipitation in midlatitude cyclones. XII: A diagnostic modeling study of precipitation development in narrow cold-frontal rainbands. J. Atmos. Sci., 41, 2949–2972.
Schultz, P. J., 1995: An explicit cloud parameterization for optional numerical weather prediction. Mon. Wea. Rev., 123, 3331–3343.
Skamarock, W. C., Klemp J. B. , Dudhia J. , Gill D. O. , Barker D. M. , Wang W. , and Powers J. G. , 2007: A description of the advanced research WRF version 2. NCAR Tech. Note NCAR/TN-468+STR, 88 pp.
Smith, B. L., Yuter S. E. , Neiman P. J. , and Kingsmill D. E. , 2010: Water vapor fluxes and orographic precipitation over northern California associated with a land-falling atmospheric river. Mon. Wea. Rev., 138, 74–100.
Tao, W.-K., Simpson J. , and McCumber M. , 1989: An ice-water saturation adjustment. Mon. Wea. Rev., 117, 231–235.
Thompson, G., Rasmussen R. M. , and Manning K. , 2004: Explicit forecasts of winter precipitation using an improved bulk microphysics scheme. Part I: Description and sensitivity analysis. Mon. Wea. Rev., 132, 519–542.
Troen, I., and Mahrt L. , 1986: A simple model of the atmospheric boundary layer: Sensitivity to surface evaporation. Bound.-Layer Meteor., 47, 129–148.
Walko, R. L., Cotton W. R. , Meyers M. P. , and Harrington J. Y. , 1995: New RAMS cloud microphysics parameterization. Part I: The single-moment scheme. Atmos. Res., 38, 29–62.
Wicker, L. J., and Skamarock W. C. , 2002: Time-splitting methods for elastic models using forward time schemes. Mon. Wea. Rev., 130, 2088–2097.
Each reanalysis anomaly field was obtained by first calculating the relevant composite mean field for 30 December 2005, and then subtracting from that mean field the long-term average for the same date for all years in the 29-year inclusive period between 1968 and 1996.
IVT is calculated from the surface to 300 hPa (see Neiman et al. 2008b for a description of the methodology).
