a. Near-surface properties

Figures 3a and 3b show time series of predicted and observed domain mean and standard deviation of near-surface variables, including daily maximum and minimum temperature, daily averaged temperature, specific humidity, and wind speed, for the summer and winter seasons, respectively. The observed daily maximum and minimum temperature values were obtained using daily summary data from the 669 COOP sites in the validating domain, and the daily averaged temperature, humidity, and wind speed were computed using hourly observations at the 193 U.S. Surface Airways stations within the domain.

In the summer season, the forecasted daily maximum temperature values are consistently lower than the observed, and minimum temperatures are always higher. Consequently, the amplitudes of the diurnal temperature cycles are smaller in the model forecasts, but the daily mean temperatures show good agreement with the observations because of the cancellation of the errors in the predicted maximum and minimum temperatures. The smaller forecasted diurnal cycles are consistent with a wet bias of 1–2 g kg⁻¹ near the model surface. Little bias is found in the forecasts of near-surface wind speeds.

Similar to the summer season, the winter forecasts of daily maximum temperature are systematically lower than the observed, but the minimum temperatures do not show a consistent warm bias as they do in the summer. The existence of a cold bias both day and night leads to a larger cold bias in the predicted daily mean temperature in winter than in the summer. Unlike the summer, the cold bias in the winter season can no longer be simply attributed to a more humid near-surface model atmosphere since the predicted humidity values appear to agree well with the observations. No systematic over- or underprediction is found in wind speed forecasting.

Several features are similar for the two seasons. First, there appears to be no clear correlation between the magnitudes of forecast errors for each variable and the magnitudes of the variable itself. For example, larger temperature forecast errors occurred during periods when observed temperatures were higher than the seasonal average, as well as periods when they were lower. The same can be said for wind and humidity forecast errors. Therefore, no conclusions can be drawn regarding whether the forecasted near-surface temperature, humidity, and winds are more accurate for a particular type of condition. Second, the standard deviations in the model forecasts are comparable to the standard deviations computed using the observations, suggesting that the forecasts captured the observed spatial variations of these near-surface variables. Finally, the forecasts adequately produced the observed day-to-day variations associated with changes in synoptic conditions, which is due primarily to the twice-daily initialization from observations that limit the error growth in the forecasting.

To quantify errors of near-surface variables, comparison statistics are computed using Eqs. (3)–(5) based on hourly observations at 193 U.S. Surface Airways stations. More than 42 600 data points for the summer and approximately 36 900 data points for the winter were used to compute the comparison statistics for temperature and wind; fewer data points were available for humidity. The results are summarized in Tables 1 and 2 for the 4- and 12-km domains, respectively. In the summer for the 4-km forecasts, the predicted temperature has a cold bias of −2.21°C during the day and a warm bias of 1.07°C at night, yielding a small overall bias of −0.57°C. The overall bias for the winter season is much larger at −2.67°C, resulting from a combination of a cold bias both day (−3.33°C) and night (−2.01°C). The humidity bias is positive during both the daytime and the nighttime, with an overall bias of 1.24 g kg⁻¹ for the summer and 0.22 g kg⁻¹ for the winter. For temperature and humidity, the biases and the error standard deviations are comparable in magnitudes, indicating that the systematic and nonsystematic errors contribute more or less equally to the total temperature and humidity errors. In both seasons, the wind speed bias is less than 0.5 m s⁻¹, and the direction bias is around 10°, which is much smaller than their corresponding error standard deviation values (1.7–2 m s⁻¹ and 35°–55°). This indicates that nonsystematic errors comprised a large portion of the total errors in the predicted winds. The statistical measures computed using the 12-km resolution forecasts are very similar to those of 4-km forecasts, suggesting that a decrease in horizontal grid spacing from 12 to 4 km does not necessarily lead to an improvement in the accuracy of forecasts of near-surface temperature, humidity, wind speed, and wind direction.

The colder and wetter MM5 forecasts are consistent with the findings from several previous studies (Manning and Davis 1997; de Arellano et al. 2001; Zhong and Fast 2003, among others). The cold and wet biases have been attributed in the past to an inadequate specification of soil moisture in the model, which is also likely to be the dominant factor here because the summer of 2003 is known to have been unusually dry in the upper Midwest and Northeast. The soil moisture values in the model, specified based on land-use categories, would not be representative for these general dry conditions. The same explanation may not apply to the winter, when the cold bias is larger and wet bias is much smaller compared to the summer.

b. Precipitation

The MM5 precipitation results were compared to the observed precipitation by interpolating modeled precipitation amounts to the 669 COOP locations in the validation domain using Eqs. (1) and (2). The precipitation validation statistics were computed using Eqs. (6)–(8) for nine successive precipitation thresholds (0.25, 2.5, 5, 10, 15, 25, 35, 50, and 70 mm day⁻¹). The statistical scores were computed separately for the summer and winter seasons and for both 12- and 4-km forecasts. The results are shown in Fig. 4.

In the summer season, the bias scores for the 12-km forecasts increase slowly from just above 2 to about 4 for the light to moderate precipitation categories and rapidly to over 10 for the heavy precipitation category. This suggests that the 12-km grid substantially overpredicts the occurrences of precipitation for all categories, especially for heavy precipitation. The 4-km grid exhibits much better precipitation forecast skill with bias values ranging from 2 in low to moderate thresholds to approximately 3 at high thresholds. For both resolutions, the threat score and the Heidke score decrease rapidly with increased thresholds. These decreases, taken together with the increase in bias score, indicate a rapid decay of precipitation predictive skill from light to heavy precipitation. Similar to the summer, the winter season statistics show poor skill in heavy precipitation forecasting for both grids, especially for the 12-km grid. However, instead of a steady decay of skill from light to heavy precipitation as in the summer, the winter season shows a decrease in the bias and an increase in the HSS score when the threshold values increased from 0.25 to 5–10 mm day⁻¹, indicating improved skill from the light to moderate precipitation categories.

A better way to examine the precipitation forecast skill is to compare the frequency of the occurrences of the four individual elements that make up the statistical scores (Fig. 5). Recall, that the four elements, A, B, C, and D, represent, for a given threshold value, correct positive, false positive, false negative, and correct negative, respectively. For all thresholds, the frequency of false positive is much higher than that of false negative, suggesting that the model has a much higher chance to produce precipitation that is absent from the observation than to miss an observed precipitation event. It is interesting to note that the frequency of forecast hits or correct positives (A) is very similar between the 4- and 12-km forecasts, although the latter exhibits much higher bias scores. The higher 12-km bias scores, defined by BS = (A + B)/(A + C), are attributed to the higher number of false positive (B) compared to false negative values (C) as shown in Fig. 5. The rapid decrease of A, B, and C values at higher precipitation thresholds with A→0 and B >> C explains the large bias scores and near-zero threat scores in the categories of heavy precipitation.

The contingency table–based evaluation statistics, although useful in measuring the model skill, provide little information about the actual errors in the precipitation prediction for a given time period. To understand the difference between the observed and predicted precipitation amount, the predicted total precipitation amount for each of the summer and winter months is compared to the observations. The results are shown in Fig. 6. The model consistently overpredicted precipitation amounts for all months, and the overprediction is much worse in the 12-km results than in the 4-km results. The monthly total precipitation amount predicted with the 4-km grid spacing is 1.3–2 times higher than the observed amounts, compared to 2–4 times higher in the 12-km predictions.

To understand whether these errors were the result of a small amount of overprediction spread throughout the season, or whether they came from a large amount of overprediction during a few isolated events, time series of domain-accumulated daily total precipitation are compared with the observed values. The comparisons are shown in Figs. 7a and 7b for the summer and winter seasons, respectively. Both plots show that the forecasted precipitation amounts were always higher than the observed except for a few days in the summer. The 12- and 4-km forecasts tend to closely track each other with the 12-km forecasted precipitation amount being persistently higher. The large error in the monthly total precipitation in June, as seen in Fig. 6, resulted primarily from the large overforecasting of the most heavy precipitation event in the summer that occurred on 23–26 June (Julian days 174–177). A similar explanation may be given to the large error in February when the 12-km predicted precipitation nearly tripled the observed amount during a heavy precipitation event that occurred on 3–4 February (Julian days 34–35). In both seasons, the forecasts appear to track the day-to-day changes of the observed precipitation reasonably well.

The overprediction in domain total precipitation at a given time can be a result of larger area coverage and/or substantially larger amount at some locations in the forecasts. To isolate these two factors, the time series of the predicted domain-accumulated daily total precipitation in Figs. 7a and 7b, which was computed using predicted precipitation at all 699 COOP sites regardless of whether precipitation was observed at the sites, is replaced by a new time series obtained using only those sites at which precipitation occurred (Figs. 8a and 8b). The predicted precipitation amount is now in very good agreement with the observations in both summer and winter seasons, indicating that the overprediction seen in Figs. 7a and 7b is due to more locations with predicted precipitation rather than excessive predicted precipitation at individual locations.

The more widespread area coverage in the precipitation prediction is also evident in the comparison of the predicted and observed area coverage of monthly total precipitation shown in Fig. 9 for June 2002 and February 2003. In February, less precipitation occurred in north-central Minnesota and southern Wisconsin and more occurred in Michigan, especially in the southeastern part of the state and in the Upper Peninsula along the southern boundary of Lake Superior. The forecasts captured this overall pattern reasonably well, but the predicted area coverage was larger than the area coverage in the observation. In June, more precipitation occurred in the western part of the domain than the eastern part, with the heaviest amount in east-central Minnesota and northern Iowa and the least amount in southern Wisconsin and most of Michigan. Although the forecast captured this general spatial distribution pattern, the heavy precipitation with amounts exceeding 140 mm month⁻¹ appears to spread over a larger area in the forecasts than in the observations.

To examine the sensitivity of predicted summertime precipitation to convective parameterizations in the model, a 10-day (26 July–5 August 2003) forecast was repeated with the Kain–Fritsch cumulus convection scheme employed in the original run replaced by the Grell scheme (Grell 1993). In the Grell cumulus parameterization, clouds are maintained by the updraft and downdraft originated at levels of maximum and minimum ambient moist static energy with no mixing between the cloud and environment except at the top and bottom of the circulations. Convection is triggered when a parcel lifted from the updraft originating level attains moist convection as determined by moist static energy. The Kain–Fritsch scheme uses a classical approach to remove convective available potential energy (CAPE) by vertical reorganization of mass. This method includes detrainment of mass and moisture from deep convective clouds and represents the exchange of mass between cloud and environment. Convection is triggered based on parcel buoyancy at the lifting condensation level. Because of differences in closure assumptions and trigger functions, large differences can often be found in the amount and spatial distribution of precipitation simulated by different convection schemes.

Figure 10 shows a comparison between domain-accumulated daily total precipitation using the two schemes with the observed precipitation for the 10-day period. Both schemes overpredicted precipitation on each of the 10 days with the Grell scheme being consistently higher. Since the forecasts were driven by the same large-scale conditions, the difference in the precipitation between the two schemes is likely to be caused by the difference in the convection-triggering mechanisms. Similar spatial distributions (not shown) were produced by the two schemes over most of the domain except near the western boundary where the Grell scheme produced significantly more precipitation. This sensitivity test, although limited to a short time period, suggests that the summer season precipitation forecasts for the current domain of the Great Lakes region are unlikely to be improved by using the Grell scheme.

c. Vertical soundings and boundary layer structure

To understand how well the model predictions were able to capture the observed vertical structure of the atmosphere, the predicted vertical profiles of potential temperature, specific humidity, and wind averaged for the summer and the winter seasons, respectively, are compared to the corresponding mean vertical profiles available twice per day at 0000 and 1200 UTC at the six upper-air sounding sites in the validating domain. The comparisons yielded similar results among the six sites, and the results are, therefore, shown in Fig. 11 for only three of the six sites.

For the mean temperature, the predicted values are lower than the observed, especially in the lowest 1500 m in the summer and 500 m in the winter, which corresponded to the average depth of the boundary layer in the two seasons. The values of the cold bias in the lower atmosphere are 2°–3°C in the summer and 1°–2°C in the winter. The cold bias is more pronounced in the afternoon (0000 UTC) than in the early morning (1200 UTC). The predicted surface-based inversion at 1200 UTC appears to be much weaker than observed in the summer season, becoming somewhat stronger than observed during winter. The predicted specific humidity profiles are wetter by 0.5–1.5 g kg⁻¹ in the boundary layer and drier above the boundary layer. The errors in the predicted wind profile appear to be larger in the winter than in the summer, and the departure of the predicted and observed wintertime winds increases with height, with winds aloft being predominantly westerly in the forecast compared to west-northwesterly in the observation. The errors in the two wind components, however, tend to cancel each other to produce total wind speeds that are in good agreement with the observed wind speeds. Some of these findings, such as the cold bias in the boundary layer, are very similar to what was found by Zhong and Fast (2003) in their comparisons of MM5 simulations with soundings taken in the Salt Lake Valley during a field campaign, although there are large differences in topography and land use between Salt Lake Valley and the Great Lakes region.

Among the various atmospheric properties produced in mesoscale forecasting, one that is important for many applications is the mixed-layer height. For example, as an input variable to air quality models, the depth of the mixed layer and the rate at which it grows have a direct impact on the mixing, transport, and dispersion of pollutants. To evaluate how well the MM5 forecasts capture the afternoon mixed-layer depths, the modeled mixed-layer depths were compared to the observations at all six upper-air sounding sites in the validation domain. The mixed-layer depths are determined as the base of the elevated inversion using predicted and observed potential temperature profiles at 0000 UTC for the summer season. The results are shown in Fig. 12 for three of the six sites. The predicted mixed-layer depths exhibit no noticeable differences among the locations, consistent with the observations. However, at all locations, the predicted afternoon mixed-layer depths are substantially lower than those observed. This underprediction is consistent with results from earlier studies (among others, Berg and Zhong 2004, manuscript submitted to J. Appl. Meteor.) showing that the MM5 predicted mixed-layer depths are very sensitive to boundary layer turbulence parameterization schemes. These previous studies focusing on a few cases concluded that boundary layer parameterization schemes in which turbulent mixing is determined based upon the predicted turbulent kinetic energy (TKE), such as the Eta boundary layer scheme employed by the current real-time predictive system, tend to underpredict mixed-layer depths. The current results support this previous finding by extending the comparison from a few cases to an entire season.

Another important boundary layer property affecting atmospheric near-surface dispersion is the surface-based radiation inversion at night. Not only are nocturnal inversions a key meteorological factor in air pollution, but they also have a great impact on agriculture and aviation because of their role in the buildup of cold air pools in lower-lying terrain and the formation of fog and frost. The MM5-predicted inversion strengths in early morning were evaluated using the 1200 UTC soundings, and the results are shown in Fig. 13. The inversion strengths were determined by the potential temperature gradients between 10 and 200 m above ground. The sounding profiles were interpolated linearly to these levels before the gradients were computed. The data points from the three sites are mixed together, indicating small spatial variation of morning inversion strengths in both the forecasts and the observations. Except for a few data points, the predicted summer inversion strengths are much weaker than those observed. It is interesting to note that in the summer, while the observed values are all positive, indicating an increase of potential temperature with height in the lowest 200 m, roughly 20% of the predicted values are below zero. These negative values represent a decrease of potential temperature with height, suggesting that a mixed layer had already developed and grown to 200 m by 1200 UTC. An examination of individual potential temperature profiles corresponding to those negative data points confirmed that a mixed layer had indeed developed in the MM5 forecasts by 1200 UTC (0700 EST), just 1–2 h after sunrise, that occurred at 0458 EST on 1 June and 0556 EST on 31 August. Figure 14 shows several of these profiles at the three sites where a mixed layer had developed in the lowest 100–200 m in the forecast while the nocturnal inversion still existed in the observation. The earlier development of a mixed layer in the forecasts is consistent with the fact that the predicted nocturnal inversion is significantly weaker than the observed, and consequently, less solar heating is required to break up the inversion before a mixed layer starts to grow. The head start of the mixed-layer growth in early morning, however, failed to produce either a deeper afternoon mixed layer or a warmer convective boundary layer, suggesting that either the surface sensible heat flux in the model is lower than that observed or the flux divergence in the boundary layer may be too high. Unfortunately, no energy flux data are available to verify these hypotheses.

d. Lake breeze

The large lakes, complex shorelines, and resulting lake breezes are unique features of the Great Lakes region. To evaluate the ability of MM5 real-time forecasts in predicting the development and evolution of the lake–land breezes, we chose to focus on six surface sites surrounding Lake Michigan on days with weak synoptic winds and clear skies when these thermally driven circulations are expected to be well-developed. The soundings from Detroit, Michigan, on the east side of Lake Michigan and Green Bay, Wisconsin, on the west side are employed to determine the weak synoptic wind days using an arbitrarily chosen criterion that observed winds at 850 mb were less than 5 kt. The days that satisfy this criterion are the same regardless of which of the two sounding sites were used, and a total of 31 days were selected from the three summer months as having weak synoptic winds. For each of the 31 days, the station cloud cover information was used to further select days with clear skies using the criterion that low- and midlevel clouds should not be present at the location. Hourly near-surface winds from days that satisfy both criteria, which range from 10 to 23 days among the six locations, were used to construct hourly mean wind vectors for a diurnal cycle obtained by vector averaging the wind at each hour over the selected days. The resulting hodographs are shown in Fig. 15 for the six sites that are almost equally distributed on the eastern and western shoreline of Lake Michigan. The hodographs in Fig. 15 are deviations of the mean wind vectors for each of the 24 h of the day from the 24-h average wind to better illustrate the diurnal nature of the winds.

The turning of wind vectors, clockwise on the eastern side and counterclockwise on the western side of Lake Michigan, is a clear indication of the development of a lake breeze in the afternoon and land breeze during night and early morning hours. The lake breezes appear to be somewhat stronger than the land breezes, especially on the eastern side of the lake, as indicated by larger vectors from the origin to the individual points on the hodograph in the afternoon hours as compared to the nighttime and morning vectors. The hodographs appear to take a more circular shape on the east side of the lake than the west, indicating that the lake and land breezes are better developed on the east side. At all sites, the model captured the general pattern of wind rotation quite well, but the predicted timing of the land to lake breeze transition and vice versa do not exactly match the observations. At some locations the modeled lake breeze appears to be slightly weaker during its peak hours in late afternoon compared to the observed speed. Overall, the model is able to predict the general development of lake–land breezes in the area surrounding the lake during the summer, although errors exist in the predicted strengths of the breezes as well as the timing of the transitions. Since the number of days satisfying our criteria is rather small in one summer, more data are needed to understand the source for the errors in lake–land breeze predictions.