Abstract

The National Centers for Environmental Prediction’s (NCEP) Eta Model, the models of the European Centre for Medium-Range Weather Forecasts (ECMWF) and the National Aeronautics and Space Administration’s (NASA) Global Modeling and Assimilation Office (GMAO) models, and the Regional Atmospheric Modeling System (RAMS) model are all examined during the Mixed-Phase Arctic Clouds Experiment (MPACE) that took place from 27 September through 22 October 2004. During two intensive observation periods, soundings were launched every 6 h from four sites across the North Slope of Alaska (NSA): Barrow, Atqasuk, Oliktok Point, and Toolik Lake. Measurements of temperature, moisture, and winds, along with surface measurements of radiation and cloud cover, were compared to model outputs from the Eta, ECMWF, GMAO, and RAMS models using the bootstrap statistical technique to ascertain if differences in model performance were statistically significant. Ultimately, three synoptic regimes controlled NSA weather during the MPACE period for varying amounts of time. Each posed a unique challenge to the forecasting models during the study period.

Temperature forecasts for all models were good at the MPACE sites with mean bias errors generally under 2 K, and the models had the fewest significant errors predicting temperature. Forecasting moisture and wind proved to be more difficult for the models, especially aloft in the 500–300-hPa layer. The largest errors occurred in the GMAO model, with significant moist biases of 40% and wind errors of 10 m s−1 or more. The RAMS, Eta, and ECMWF models had smaller moist biases in this layer. Both the Eta and RAMS models overestimated the surface incident shortwave radiation, underestimated longwave radiation, and underestimated cloud cover fraction. Overall, the bootstrapping results coincided with findings from conventional statistical comparisons as model outputs with the largest errors were most likely to be captured and declared statistically significant in the bootstrapping process.

The significant model errors during MPACE were predominantly traced to the inability of the models to simulate disturbances in synoptic regime I, warm or cold biases over higher inland terrain, a warm bias along the NSA coastal waters in the Beaufort Sea, and difficulty in forecasting the intensity of the explosive cyclone in synoptic regime III.

1. Introduction and motivation

The Arctic environment presents difficult challenges for forecasters and models at all scales, spanning high-resolution cloud models to general circulation models (GCMs). Quantifying the interactions between clouds, aerosols, the atmosphere, sea ice, the open ocean, and radiation remain a daunting task. Further complicating the scenario is the fact that large-scale synoptic conditions drastically change how these physical components interact with each other. Synoptic conditions determine Arctic surface wind fields that influence sea ice extent and movement (Overland and Pease 1982). Herman and Goody (1976) and Curry and Herman (1985a,b) showed that synoptic conditions strongly influence cloud cover, thickness, and type. In addition to regulating small-scale processes, synoptic-scale systems transport tremendous amounts of heat and moisture from lower latitudes into the Arctic and are pivotal to restoring the global energy balance (Sanders and Gyakum 1980; Gyakum et al. 1989; Bullock and Gyakum 1993). If this advection of heat and moisture were to cease, temperatures in the Arctic would drop by as much as 50°C (Steffen and DeMaria 1996).

Not surprisingly, sea ice retreat over recent decades, culminating with the record minimum experienced in the summer of 2007 (Parkinson et al. 1999; Partington et al. 2003; Serreze et al. 2003; Rothrock et al. 2003; Lindsay et al. 2009), has coincided with increased heat transport into the Arctic through more frequent synoptic activity and additional warming from increased cloud cover in the Arctic (Serreze et al. 1993; Key and Chan 1999). Recent field projects such as the First International Satellite Cloud Climatology Project (ISCCP) Regional Experiment (FIRE) Arctic Cloud Experiment (ACE; Curry et al. 2000), the Coordinated Eastern Arctic Research Experiment (CEAREX), the Lead Experiment (LeadEx; Fett et al. 1994), and the Surface Heat Budget of the Arctic Ocean Experiment (SHEBA; Uttal et al. 2002) have collected valuable measurements for researchers, and future field experiments will continue to broaden our understanding of the Arctic environment. Nonetheless, data in the Arctic are sparse and detailed understandings of many Arctic processes are currently lacking.

While Arctic cyclones have been investigated extensively (Keegan 1958; Overland and Pease 1982; Serreze and Barry 1988; Yarnal and Henderson 1989; Serreze et al. 1993; Zhang et al. 2004), studies on how accurately current forecasting models can predict these systems and synoptic conditions in the Arctic are rare. Beesley et al. (2000) compared output from the European Centre for Medium-Range Weather Forecasts (ECMWF) model to observations during the SHEBA experiment in 1997. They found that the model had a warm bias at the surface and underestimated clouds in the 500–3000-m layer. More recently, Bromwich and Wang (2005) compared 15- and 40-yr ECMWF reanalyses to rawinsonde data from CEAREX and LeadEx. CEAREX was conducted from September 1988 through May 1989 near Svalbard Island north of Scandinavia, while LeadEx took place in the Beaufort Sea spanning 16 March–25 April 1992. Overall, the ECMWF model did an excellent job predicting 500-mb height fields in the Arctic. The model predicted temperatures well, though it exhibited a slight warm bias for both periods. Wind forecasts were highly variable in accuracy. During the LeadEx period, the 40-yr ECMWF reanalyses did exceptionally well as their model wind speeds were within 8% of the observed winds in the boundary layer and within 2% of the observed winds at or above the 700-hPa level (Bromwich and Wang 2005). During CEAREX, however, model performance was poor with consistent overestimations of both u- and v-wind components of 35%–65%. Wind speeds were overestimated by as much as 75% in some cases.

In this paper we build upon prior studies of model performance in the Arctic by assessing the performance of four models during the Mixed-Phase Arctic Cloud Experiment (MPACE), including cloud and radiative properties in the comparisons where possible. Differences between model outputs and observations were interpreted within prevailing synoptic conditions to determine the physical causes of model performance problems. MPACE was conducted across the North Slope of Alaska (NSA) from 27 September through 22 October 2004, and was funded by the Department of Energy’s (DOE) Atmospheric Radiation Measurement (ARM) program (Verlinde et al. 2007). The goal was to study the microphysics, radiative properties, thermodynamics, dynamics, and life cycle of mixed-phase clouds in the Arctic through an arrangement of observation stations and soundings along the coastline of the NSA (Fig. 1).

Fig. 1.

Map of the MPACE experimental domain and surrounding area. The rawinsonde launch sites were located at Barrow, Atqasuk, Oliktok Point, and Toolik Lake.

Fig. 1.

Map of the MPACE experimental domain and surrounding area. The rawinsonde launch sites were located at Barrow, Atqasuk, Oliktok Point, and Toolik Lake.

2. Observational data and model output

This section describes the data gathered during MPACE. In addition to the radiosondes launched at the four MPACE sites of Barrow, Atqasuk, Oliktok Point, and Toolik Lake, additional instruments were run at Barrow and Atqasuk to measure radiation and cloud properties. At Barrow, the DOE’s ARM program operated cloud radar and lidar to quantify the locations and thicknesses of clouds, along with radiometers to measure surface incident shortwave and longwave radiation during MPACE. Radiometer readings were also available at Atqasuk during the experiment (Fig. 1). Output from the forecasting models used in the intercomparisons was acquired and its properties are discussed in detail.

a. Observational data

Vaisala RS-92 radiosondes were launched every 6 h at all four NSA sites during two intensive observation periods (IOPs) covering 4–9 and 14–22 October 2004. Outside of this period, sondes were launched sporadically between 26 September–3 October and 9–14 October 2004. Studies evaluating the performance of previous-generation RS-90 radiosondes at Barrow demonstrated that the temperature readings were accurate to within 1°C and the relative humidities to within 5% (Mattioli et al. 2007).

Several measurement systems acquired data on atmospheric properties above, within, and below cloud layers. High spectral resolution lidar provided by the University of Wisconsin (Eloranta 2005) and depolarization lidar from the University of Alaska Fairbanks (Sassen 1994) were used in conjunction with ARM Millimeter-Wavelength Cloud Radar (MMCR) and radiometer data to gather information about cloud heights, thicknesses, and radiative properties.

b. Model output

Four forecasting models were used in the intercomparison during MPACE. The ECMWF and National Aeronautics and Space Administration’s (NASA) Global Modeling and Assimilation Office (GMAO) models provided the temperature, relative humidity, and wind fields used in this study. In addition, instantaneous cloud cover and radiation fields were also available from the National Centers for Environmental Prediction’s (NCEP) Eta Model and the Colorado State University Regional Atmospheric Modeling System (RAMS) model. Since radiation measurements were limited to Barrow and Atqasuk, and radar and lidar cloud measurements were limited to Barrow, comparisons were restricted to these sites when assessing model performance for these quantities.

Table 1 summarizes the vertical levels and horizontal spatial resolutions for the model data that we used in our analysis. In some cases (e.g., the vertical levels from ECMWF), the data represent only a subset of the complete model fields. The Eta and ECMWF models each had grid points within 20 km of the four MPACE sounding locations, with the closest grid point being selected for comparisons at a particular site. The GMAO grid points were within about 50 km of the MPACE sites. We chose the nearest model land grid point for each site in the comparisons, since choosing a grid point over the ocean would contaminate the results. All of the models had data available at the following seven vertical levels—1000, 925, 850, 700, 500, 300, and 100 hPa—though most had considerably more levels that were used in this study.

Table 1.

Vertical levels and horizontal spatial resolutions of the data available for the analysis and forecast hours. The vertical levels used in the analysis are shown, while the numbers of levels in the entire model are shown in parentheses. Cloud and radiation data were available for two of the models used in this study.

Vertical levels and horizontal spatial resolutions of the data available for the analysis and forecast hours. The vertical levels used in the analysis are shown, while the numbers of levels in the entire model are shown in parentheses. Cloud and radiation data were available for two of the models used in this study.
Vertical levels and horizontal spatial resolutions of the data available for the analysis and forecast hours. The vertical levels used in the analysis are shown, while the numbers of levels in the entire model are shown in parentheses. Cloud and radiation data were available for two of the models used in this study.

The Eta Model (Black 1994; Rogers et al. 1996) was implemented into operational forecasting in June 1993 (Riphagen et al. 2002). The model featured an eta vertical coordinate that was both pressure based and normalized, allowing the model to better simulate atmospheric flow in areas with significant terrain than did previous models (Black 1994). A complete description of the model, including boundary conditions and other technical information, can be found in Mesinger et al. (1988), Black (1994), and Tarasova et al. (2006). For this study we used output from the Eta grid 216, which covers the entire state of Alaska and the Aleutian Islands extending northward into the central Arctic Ocean and southward into the northern Pacific Ocean. The vertical and horizontal resolutions of the Eta data available to us are outlined in Table 1.

Radiation parameterizations were adopted from Lacis and Hansen (1974) and Fels and Schwarzkopf (1975). Recent upgrades to the model radiation and land cover schemes are detailed in Riphagen et al. (2002), Betts et al. (1997), and Black et al. (1997). The grid-scale cloud cover fraction is now parameterized as a function of relative humidity and cloud water or ice mixing ratio (Tarasova et al. 2006; Xu and Randall 1996; Hong et al. 1998). The convective cloud fraction is dependent on precipitation rates (Slingo 1987). The extinction coefficient for clouds depends on the prognostic cloud or ice water ratio in the model (Tarasova et al. 2006).

The RAMS model was created in the early 1980s by the merging of three separate models (Cotton et al. 2003). The model has the distinct advantage of allowing the user to select from various resolutions and parameterizations, providing a wide range of applications (Cotton et al. 2003). In our study, data available to us were from runs with 4-km horizontal resolution, as this version of RAMS was originally intended to capture the boundary layer Arctic stratus clouds found along the coast of the NSA during MPACE. While this is much smaller than the other models in this study, we thought it might be useful to gauge the performance of a small-scale model against larger synoptic-scale models for the region. As one might expect with a 4-km model, the domain for this simulation was small, about a 220-km-sided square centered over the NSA. This did not include Toolik Lake, so the site was left out of RAMS intercomparisons for the study. In the vertical, the model output had 11 levels that were used in this study. The radiation parameterization is a two-stream scheme (Harrington 1997; Harrington et al. 1999) that includes the interaction of three solar and five infrared bands with model gases and cloud hydrometeors. The cloud microphysics is a two-moment bulk scheme outlined in Meyers et al. (1997). Boundary conditions are outlined in Klemp and Wilhelmson (1978).

The ECMWF model became operational in August 1979 in order to provide midrange forecasts to companies that required them (ECMWF 2008a). Since then, the model has undergone numerous upgrades and changes, including a four-dimensional variational data assimilation system (ECMWF 2008b). At the time of our study we used the T511 version of the model, which used a Gaussian grid described by Hortal and Simmons (1991) and Simmons and Burridge (1981) with an average horizontal grid spacing of slightly less than 40 km. The domain for our study was quite large, encompassing 55°–85°N and 140°E–120°W. The vertical grid at the time had 60 vertical levels using a sigma coordinate system between the surface and 70 km (ECMWF 2007a), though only 7 levels were available for our comparisons. Boundary conditions and other technical information are available from ECMWF (2007b). Key upgrades in 2004 included snow analysis using the National Environmental Satellite, Data, and Information Service (NESDIS) snow cover product, and improved winds (ECMWF 2008b). The snow cover analysis is of particular interest since the study takes place during autumn in the Arctic.

The GMAO model was originally developed in the late 1980s and early 1990s for the purpose of simulating the transport of chemical constituents and water vapor in the atmosphere (Lin 2004; Rood 1987; Allen et al. 1996; Lin et al. 1994). The model features a finite-volume dynamical core described in Lin (2004) using a sigma vertical coordinate. The GMAO had 36 vertical levels extending up to 0.2 hPa and a horizontal resolution of 1.00° latitude × 1.25° longitude with a domain spanning the globe at the time of our study. Levels above 50 hPa were excluded from the analysis since they were not relevant for this study; hence, only 25 levels were actually used. The model cloud parameterizations are detailed in Norris and da Silva (2007) and Slingo (1987), using a critical value of relative humidity to determine cloud cover. The boundary conditions and other technical information are detailed in Lin (2004) and Norris and da Silva (2007).

3. Methodology

Model forecasts of temperature, moisture, and wind profiles from the Eta, ECMWF, GMAO, and RAMS models were compared to sounding observations at the four MPACE surface sites. The statistical significances of model mean errors were tested using the block bootstrap statistical technique outlined by Efron and Tibshirani (1991), Wilks (1997), and Marchand et al. (2006). Model errors were examined as a function of height and time, with each model’s performance evaluated throughout the MPACE period. The synoptic and mesoscale conditions during MPACE were also examined to interpret model errors illuminated by the block bootstrap method.

a. Why bootstrap?

In most standard statistical tests (e.g., the t test), it is assumed that all data points are independent of each other and that the underlying distribution of the parent population is known. This is a fair assumption to make in some instances, but not in meteorology. Many meteorological quantities, such as temperature, moisture, and winds, exhibit strong spatial and temporal correlations. A simple example would be a frontal passage in which temperatures for both observations and model outputs are highly spatially and temporally correlated ahead of and behind the front. In addition, the parent population of meteorological data for a given location at every height and time is unknown. One way to test collections of meteorological data for differences is the bootstrap approach, which does not make assumptions about data independence or the underlying distributions.

b. The bootstrap statistical technique

We start with two datasets Y and Z of lengths ny and nz, which in this case will correspond to the model outputs and observations for a given variable. The goal is to determine if the distributions of the data in Y and Z are such that they share a common parent population, which will be our null hypothesis. Therefore, we assume they are from the same parent population and test whether this hypothesis is unlikely. If there is a statistically significant difference between Y and Z at the 80% confidence interval, we can reject the null hypothesis and say that with 80% certainty Y and Z do not come from the same parent population.

To implement the bootstrap test, we first combine sequentially datasets Y and Z into a new dataset X, where X = YZ. We then take random samples of length L from X and create two new datasets Y* of length ny and Z* of length nz. Resampling is done with replacement, meaning that after elements are selected from X they remain within X and can be selected again. The resampling process begins by choosing a random sample location xi from dataset X. Using the location of xi in X as a starting point, we take a block of elements of length L and place them into Y*. We repeat this process until Y* is of length ny. The resampling process resumes in the same manner, shuffling blocks of length L into Z* until its length is nz. Using the datasets Y* and Z*, we proceed to calculate the standard difference d*:

 
formula

where

 
formula

Repeating this process a large number, nb, of times produces a Gaussian distribution of d* values. The difference statistic d for the original datasets Y and Z is now calculated in the same way and compared to the distribution of d* to determine if d is an unlikely value. If d falls outside the 80% confidence interval of d*, we conclude that the two datasets are significantly different and reject the null hypothesis. If d falls within the 80% confidence interval of d*, then we do not reject the null hypothesis and can claim that the two datasets may come from the same parent population (Wilks 1997; Marchand et al. 2006).

For the bootstrap simulations in this study, we used a block length L = 2. The number of resamplings (nb) was set at 10 000 for all bootstrapping comparisons. The small block length choice was based on the number of soundings available (41–57) for each site, leading to sample sizes that were much smaller than the sample sizes analyzed in Marchand et al. (2006), where variable block lengths up to 6 were used. When we repeated the analysis with a block length of 4, the results were skewed by the fact that virtually no errors were judged to be statistically significant—not even RH errors as high as 50% or wind errors that were larger than the magnitude of the wind itself. This is because the block length was fairly large in relation to the sample size creating more outliers and a higher standard deviation within the resampling, thereby stretching the confidence intervals. Thus, only errors that are astronomical in magnitude have a chance of being captured. This eliminated the possibility of using a block length of 4 or larger. Using a blocklength of 2 yielded more reasonable results that were similar to bootstrapping resamplings using single values.

The choice of the confidence interval (80%) also has its roots in the small sample size of the dataset in this study. The small sizes of the data samples here, especially within individual synoptic regimes, precluded the use of high confidence intervals. When the results of our bootstrap analyses at the 80% confidence level were compared to those from a 99% t test, we rejected the null hypothesis far less often for the bootstrapping test. Many of the errors picked up by the t test were quite small in magnitude, such as temperature errors of a few tenths of a degree, indicating the test is too permissive (rejects the null hypothesis too often). While this result was surprising given the difference in confidence intervals used here, we believe this illustrates the superiority of the bootstrapping method itself for this application, as the results from bootstrapping using a low confidence interval were a better indicator of model performance than using maximum confidence from a more traditional method based on Gaussian statistics.

4. MPACE synoptic weather events

The fall 2004 transition during MPACE was marked by periods of subfreezing temperatures, explosive cyclogenesis (Sanders and Gyakum 1980), and frequent winds over 15 kt. One of the defining factors that made the fall transition season during MPACE unique was the fact that 2004 was a high melt year. Historically, pack ice would have advanced to the coastline prior to, or during, the early stages of MPACE. Instead, the period was marked by the presence of open water along the coastline during the entire period, allowing relatively warm, moist air to be readily available. Figure 2 shows a time series of the observed temperatures at Barrow during MPACE along with the climatological means for the period. Not surprisingly, the study period was marked by above average temperatures. Sea ice did not reach the Arctic coastline of Alaska until 31 October, approximately 1 week after the study ended. During MPACE, the NSA was characterized by three distinct synoptic regimes (Verlinde et al. 2007).

Fig. 2.

Time series showing temperatures (solid) and climatological high and low temperatures (dashed) at Barrow, during MPACE. The two solid vertical lines separate synoptic regimes I–III.

Fig. 2.

Time series showing temperatures (solid) and climatological high and low temperatures (dashed) at Barrow, during MPACE. The two solid vertical lines separate synoptic regimes I–III.

a. Synoptic regime I

The first synoptic regime controlled weather over the NSA from 23 September to 1 October and was characterized by unsettled conditions. An upper-level trough over the Chukchi Sea extending into western Alaska combined with a ridge over the central Pacific Ocean to funnel several short-wave systems into the NSA. The first short wave originated over the Aleutian Islands and was steered to the northeast into central Alaska by the upper trough. The system turned north on 23 September, moving along the Alaskan–Canadian border toward the Arctic Ocean. A secondary low developed early on 24 September near Deadhorse, Alaska, just east of Oliktok Point, and deepened to 984 hPa as it drifted westward and stalled about 85 km south of Barrow by 1800 UTC. The storm remained virtually stationary over the next 24 h before slowly moving northeast and dissipating on 26 September.

The upper-level trough broke down into a cutoff low as it moved into central Alaska during this period. On 26 September, a second short wave moved onshore near Anchorage and promptly moved north-northeastward toward the Arctic Ocean. As the disturbance neared the coastline, it spawned a secondary low in the Beaufort Sea near Deadhorse that turned westward toward Barrow early on 27 September. The upper-level trough in central Alaska drifted northward and became vertically stacked with the surface low just south of Barrow that afternoon. On 28 September, the weakening 500-hPa trough continued to drift northward, and westerly steering currents moved the system eastward out of the area. Deep frontal clouds dominated Barrow, Atqasuk, and the western NSA on 27 and 28 September. A dry slot kept Oliktok Point and Toolik Lake nearly cloud free on 27 September before a return to deeper clouds on 28 September as the system departed the area.

The stormy pattern continued as a 990-hPa low moved northeastward from the Gulf of Alaska and came onshore over southwestern Alaska near Bethel on 29 September. A new cutoff low moved in to the west of the surface low, pushing the system northeastward toward the Alaskan–Canadian border. The system continued its track on 30 September, bringing scattered upper-level clouds to Oliktok Point and Toolik Lake as it passed through the eastern NSA. During the afternoon of 30 September, the upper-level trough accelerated eastward, quickly pulling the final disturbance of synoptic regime I out of the area by the morning of 1 October. Remnants from an old frontal system over Siberia moved eastward toward the NSA, supplying upper-level moisture in addition to the boundary layer clouds that were in place during 30 September and 1 October. On 2 and 3 October, the upper-level pattern began to change dramatically with the trough breaking down and moving off to the east. In its place a strong surface high moved into the NSA (Fig. 3) with a broad ridge building over central Alaska. This ridge strengthened considerably on 4 October, establishing synoptic regime II.

Fig. 3.

Mean sea level pressure (contours) and surface winds (arrows) over the NSA at 1200 UTC 2 Oct 2004.

Fig. 3.

Mean sea level pressure (contours) and surface winds (arrows) over the NSA at 1200 UTC 2 Oct 2004.

b. Synoptic regime II

The second synoptic regime was firmly established on 4 October and lasted through 14 October. Unlike synoptic regime I, where a deep trough was in place across the NSA and storm systems were permitted to enter the NSA, there was now a strong surface high in place that deflected disturbances away from the NSA. On 4 October, this high was in place northeast of Barrow over the Arctic Ocean. An upper-level ridge also established itself over southeastern Alaska and the Yukon, extending northwestward toward Barrow. The subsidence associated with this system allowed for partial clearing along the coastal NSA and adjacent Arctic Ocean instead of the overcast conditions that typically predominate. More frequent breaks in the Arctic stratus over the pack ice allowed for more efficient radiative cooling and further intensification of the surface high. This produced stronger subsidence and clearer skies that allowed for even more efficient cooling. As a result, the high remained over the pack ice and strengthened considerably as temperatures in that region dropped from −5°C during the morning of 6 October to −25°C by 13 October.

During this period one small midlevel disturbance influenced the NSA. This disturbance appeared initially during the afternoon of 5 October, just south of the Brooks Range, as a small wave in the 700–500-hPa height fields with a small vorticity maximum. As the disturbance was steered north over the Brooks Range, it spun up and intensified as it descended on the lee side. Moisture associated with this system wrapped around the disturbance as it occluded, reaching Barrow and Deadhorse on 6 October. By 7 October the system was dissipating and relinquished its influence on the NSA shortly thereafter.

For the vast majority of the period, flow associated with the high pressure system came from the east or east-northeast with considerable fetch along the Arctic Ocean before impinging on the Alaskan coast. Boundary layer clouds originating off the pack ice kept coastal regions consistently overcast, and the weather conditions from 9 to 14 October were remarkably uniform across the NSA.

On 15 October, however, the surface high began to migrate southeastward toward Canada. This marked the end of synoptic regime II, as storm systems would once again influence the NSA. Overall, synoptic regime II was marked by above average temperatures for the first half of the period and more seasonal temperatures during the second half associated with the weak trough. The persistent boundary layer stratus clouds also made for exceptionally low diurnal temperature ranges along the coast as 9 of 11 days at Barrow had diurnal ranges of 2°C or less.

c. Synoptic regime III

Synoptic regime III dominated the NSA from 16 to 22 October. The centerpiece of this regime was an intense and fast-developing low pressure center (940-hPa peak strength with a 42-hPa drop analyzed over 24 h) that formed near Kamchatka, Russia, on 17 October and propagated northward through the Bering Strait into the northwestern portion of the Chukchi Sea from 18 to 22 October. Once the surface high that was present during synoptic regime II moved out of the area, a couple of weak fronts associated with low pressure systems well to the north in the central Arctic Ocean moved through on 16 and 18 October. In the Bering Strait a much stronger low pressure system was developing during this period and subsequently became the dominant feature for the remainder of the period. The accompanying 500-hPa trough brought geopotential heights below 5000 m for the first time in the Northern Hemisphere fall 2004 season. The main frontal system associated with this low pressure center passed through Barrow on 19 October.

With a strong low pressure center to the southwest and west of the NSA bringing a vigorous southerly flow, surface temperatures at Barrow were unseasonably warm throughout synoptic regime III. Temperatures prior to the frontal passage were around −5°C with a high of −3°C on 18 October despite the dramatic change in surface winds from the southeast at 5 kt during the morning hours to east at 24 kt by midnight. On 19 October, strong winds continued during the early morning hours, but eased quickly with the frontal passage during the midmorning to early afternoon hours. With thick clouds blanketing the region, temperatures rose slightly with a high of 0°C and a low of −7°C. A day after the frontal passage, surface temperatures were comparable to those seen prior to the frontal passage, remaining near −5°C and peaking at −4°C. Winds became light and variable. Two small secondary systems spawned by the low moved through the area on 20 and 21 October, bringing cooler air into the NSA. The infusion of cold, dry air aloft combined with the frontal lifting created a more unstable environment that produced snow showers on 21 and 22 October. Overall, synoptic regime III was marked by a mean temperature of over 5°C above average, the warmest anomaly of the three synoptic regimes.

d. Discussion

In comparing the MPACE synoptic regimes with previous data in the Arctic, most of what was experienced during MPACE was fairly typical. Regime II coincides with findings by Serreze et al. (1993) and Serreze and Barry (1988) of a strong, dominant surface high north of Barrow. This high can be found year-round, but is strongest in the winter. Synoptic regime III followed results from a fall transition study by Overland and Pease (1982) as a particularly intense cyclone developed near the western Aleutian Islands, then propagated north through the Bering Strait. As the storm tracked northward, southerly winds to the east of the system across the NSA and Arctic Ocean prevented the sea ice from advancing, consistent with observations from Overland and Pease (1982) in the Bering Strait. This track is not uncommon, as the Bering Strait and eastern Siberia are regions in the Arctic where cyclones predominantly move northward according to Serreze et al. (1993). Most systems in the Arctic tend to move west to east in a zonal fashion, gradually curving northeastward as time progresses. While the system’s track was not unusual, its strength (940 hPa), especially for an October storm, was remarkable. Regime I saw several disturbances move through the region originating from the northern Pacific Ocean, which is common. However, the secondary low development along the coast and the westward motions of some of the disturbances do not appear to happen very often and could prove to be an interesting case study in future work. Nonetheless, it seems logical that secondary low development would occur in the early fall over this region. During this time, the mild coastal waters of the Arctic Ocean contrast sharply with the rapidly cooling Alaskan mainland to the south and bitterly cold advancing pack ice to the north, creating a rich baroclinic zone for cyclone development, if only for a few weeks before the Arctic Ocean freezes over.

5. Results

Using the bootstrapping method, temperatures, relative humidities, and u- and υ-wind components from the Eta, ECMWF, GMAO, and RAMS models were compared to observations at Barrow, Atqasuk, Oliktok Point, and Toolik Lake within synoptic regimes II and III, and for the entire MPACE period. Unfortunately, only three soundings were launched during synoptic regime I. While these were included in the MPACE comparisons, this is an insufficient number for an individual regime intercomparison. The surface pressure at Toolik Lake varied from 910 to 935 hPa during MPACE with 18 of 41 soundings having pressure readings at, or greater than, 925 hPa. Therefore, 925 hPa was the lowest vertical level that comparisons were performed at Toolik Lake. In addition, we mapped out each model’s respective analysis and 24-, 48-, and 72-h forecasts in an attempt to get a physical picture of what the models were doing wrong throughout the MPACE period and specifically in synoptic regimes II and III.

The forthcoming sections showcase the significant errors for each model. Model variables that had significant errors in half or more of their forecasts at a particular level are shown in the summary graphs accompanying each subsection, except for the RAMS case, where significant errors that were found in 40% or more of its forecasts are posted. While we realize this is a somewhat more stringent test for RAMS, keep in mind that the RAMS model only goes out to 48 h whereas the other models go to 84 h or more. We did not want to penalize the other models simply for having longer forecasts, which would be more prone to having errors. This filters out errors based on one bad forecast at one level and time, and illustrates more consistent errors that we feel should be examined. As with any statistical test, one must consider cases where the null hypothesis is rejected when it should not be and vice versa. However, the likelihood of having errors of this type occurring half the time in sets of 8–10 forecast times (5 for the RAMS) is extremely minute.

a. Eta Model performance

The Eta Model predicted temperatures, relative humidities, winds, radiation, and cloud fields across the NSA during MPACE out to 84 forecast hours. Figure 4 shows the vertical distribution of the significant temperature, relative humidity, and wind errors for the Eta Model at Barrow, Atqasuk, Oliktok Point, and Toolik Lake during MPACE. Levels where the Eta Model had both significant positive and negative errors are referred to as varied biases in Fig. 4. A significant cold bias of 1 K was present at Toolik Lake from the surface up to 800 hPa throughout MPACE with values of up to 2 K during synoptic regime II. A warm bias of 1 K at 975 hPa can be seen at Oliktok Point during synoptic regime III. Higher aloft, statistically significant cold biases of 1–2 K were found at Barrow, Oliktok Point, and Toolik Lake in the 200–175-hPa layer during MPACE. Greater cold biases up to 3 K were found at Toolik Lake, while warm biases of the same magnitude were found at Barrow during synoptic regime II. At 50 hPa, even larger warm biases of 4–6 K were found at Barrow, Atqasuk, and Oliktok Point throughout MPACE. These stratospheric biases were due to the model placing the tropopause lower in the atmosphere than it actually was, and also warming too quickly in the stratosphere. The Eta Model often warmed about 5°C between 100 and 50 hPa while most soundings showed essentially an isothermal atmosphere in this layer.

Fig. 4.

Statistically significant [80% confidence intervals (CI)] temperature, relative humidity, and u- and υ-wind component errors for the Eta Model at the four MPACE sites. The horizontal widths of the error bars are proportional to the magnitudes of the errors with sample bars given on the right to assist the reader.

Fig. 4.

Statistically significant [80% confidence intervals (CI)] temperature, relative humidity, and u- and υ-wind component errors for the Eta Model at the four MPACE sites. The horizontal widths of the error bars are proportional to the magnitudes of the errors with sample bars given on the right to assist the reader.

In predicting relative humidity, the Eta Model had significant moist biases of 10%–15% at all four sites in the 600–200-hPa layer during MPACE. The inland sites (Atqasuk and Toolik Lake) had more substantial errors during synoptic regime II. Significant moist biases of 10%–15% were found in the 1000–800-hPa layer at the western NSA sites (Barrow and Atqasuk), while significant dry biases of the same magnitude appeared at the eastern sites (Oliktok Point and Toolik Lake). For wind forecasts, significant westerly biases of 3–5 m s−1 were found in the u-wind components at Barrow, Atqasuk, and Oliktok Point in the 600–300-hPa layer throughout MPACE with larger errors being found during synoptic regime III. At Toolik Lake significant easterly biases of 2–4 m s−1 were found from the surface (925 hPa) up to 600 hPa. In examining the v-wind component, significant southerly biases of 1.5–4.5 m s−1 were present in the 975–775-hPa layer during MPACE at all four sites with larger errors occurring in synoptic regime III. Significant northerly biases of similar magnitude were also found in the stratosphere above 200 hPa. Overall, wind error magnitudes showed no tendency to increase with height. This was a bit surprising given the fact that winds at 250 or 300 hPa are sometimes an order of magnitude greater than those near the surface. This also points to the source of errors as inaccuracies in direction rather than magnitude for the Eta Model.

Figure 5 shows a time series of the Eta average forecast cloud fraction, and downwelling shortwave and longwave radiation, at Barrow along with observed values during MPACE. Figure 6 shows Eta average forecasts of downwelling shortwave and longwave radiation versus observations together with errors in the model radiation versus errors in the model cloud fractions at Barrow. The Eta Model generally overestimated incoming shortwave radiation. On average, the Eta Model significantly overestimated incident shortwave radiation at the surface by about 30 W m−2 at Barrow and Atqasuk. This is consistent with results from Yucel et al. (1998), who showed the model had a tendency to overestimate incoming shortwave radiation at the surface, particularly on cloudy days, which are common on the NSA. Conversely, incident longwave radiation was significantly underestimated for all forecasts, though errors at Barrow (25–40 W m−2) were consistently smaller than those found at Atqasuk (45–55 W m−2) by 15–20 W m−2. In calculating the total radiation at the surface, the Eta Model’s positive shortwave radiation bias and negative longwave radiation bias tended to cancel out, leaving only a handful of significant errors. Given that total radiation values were around 300 W m−2, the Eta Model appeared to perform well, though this was strictly by the coincidence of having two significant errors cancel one another.

Fig. 5.

Time series of the Eta average forecast cloud fraction, downwelling shortwave radiation, and downwelling longwave radiation along with observed values at Barrow, during MPACE.

Fig. 5.

Time series of the Eta average forecast cloud fraction, downwelling shortwave radiation, and downwelling longwave radiation along with observed values at Barrow, during MPACE.

Fig. 6.

(left) Eta average forecast downwelling shortwave and longwave radiation vs observed values at Barrow, during MPACE. (right) Eta shortwave and longwave radiation errors (relative to observations) vs Eta cloud fraction errors (also relative to observations) at Barrow, during MPACE.

Fig. 6.

(left) Eta average forecast downwelling shortwave and longwave radiation vs observed values at Barrow, during MPACE. (right) Eta shortwave and longwave radiation errors (relative to observations) vs Eta cloud fraction errors (also relative to observations) at Barrow, during MPACE.

An analysis of the cloud fields showed the Eta Model underpredicted cloud cover by 11.8% during MPACE (Fig. 5). For synoptic regimes I–III, the model underpredicted cloud cover by 8.9%, 12.2%, and 18.6%, respectively. The overall differences during MPACE and during synoptic regime III were statistically significant. Interestingly, the Eta Model overestimated relative humidity, particularly in the middle troposphere, yet underestimated cloud fraction. A comparison of radiation and cloud fraction errors (Fig. 6) for the Eta Model illustrates some overestimation of downwelling shortwave radiation and underestimation of downwelling longwave radiation correlating with cloud underestimation, though there was significant variability in both the shortwave and longwave radiation errors when cloud fraction errors remained small.

During synoptic regime I, the model had great difficulty tracking the first two disturbances of this period. These disturbances moved north through interior Alaska to the Arctic Ocean and spawned a secondary cyclone in the Beaufort Sea that initially moved westward. The model could not accurately capture the redevelopment and westward motion of the secondary disturbances along the NSA coastline after about 24 h.

Many of the biases found during synoptic regime II were also found for the entire MPACE period. This is not surprising given that atmospheric conditions were nearly uniform for the entire period, so errors here were likely to be indicative of natural model biases inherited from flaws in the model parameterizations. Over the entire MPACE period, the Eta Model had a consistent cold bias over the elevated interior and the Brooks Range. This translated into the significant cold and dry biases at Toolik Lake. The resulting negative height bias contributed to the cyclonic (easterly) bias found there. The Eta Model also had a pronounced warm bias in the NSA coastal waters that may have contributed to the moist biases found at Barrow throughout MPACE.

During synoptic regime III, the Eta Model tracked the explosive low well, but overestimated the intensity and pressure gradient over the NSA, which directly caused the southerly wind bias found at all sites during this period. The Eta Model also overestimated the areal extent of the frontal cloud shield associated with the low.

b. ECMWF model performance

The ECMWF model predicted temperatures, relative humidities, and wind fields across the NSA during MPACE out to 84 forecast hours. Significant errors at the four MPACE sites are illustrated in Fig. 7. Temperature forecasts for the coarse-resolution ECMWF model had mean bias errors less than 1 K in magnitude from the analysis out to 84 h for all sites, except Toolik Lake where significant cold biases of 1.5–2.0 K were found at 850 hPa. For relative humidity, a few significant biases of under 10% were found at 1000 hPa for Barrow, Atqasuk, and Oliktok. Larger dry biases of 12%–15% were present during synoptic regime II at Toolik Lake at 700 hPa and Barrow at 800 hPa.

Fig. 7.

Statistically significant (80% CI) temperature, relative humidity, and u- and υ-wind component errors for the ECMWF model at the four MPACE sites. The horizontal widths of the error bars are proportional to the magnitude of the errors with sample bars given on the right to assist the reader.

Fig. 7.

Statistically significant (80% CI) temperature, relative humidity, and u- and υ-wind component errors for the ECMWF model at the four MPACE sites. The horizontal widths of the error bars are proportional to the magnitude of the errors with sample bars given on the right to assist the reader.

Significant u-wind errors were concentrated at altitudes below 850 hPa, where significant easterly biases of 2–4 m s−1 were found at Barrow, Atqasuk, and Toolik Lake. Overall, the largest u-wind mean errors were found at 925 hPa during synoptic regime II at Toolik Lake. For the υ-wind component, significant southerly biases of up to 6 m s−1 were found at midlevels at Barrow (500 and 300 hPa) and Oliktok Point (700 and 500 hPa) during synoptic regime III. Statistically significant northerly biases of 3–4 m s−1 were also found near the surface (925 hPa) at Toolik Lake for the entire MPACE period and during synoptic regime II. The model performance was clearly inferior during synoptic regime III when southerly biases of up to 10 m s−1 could be found in some of the long-range forecasts. Overall, these findings are in agreement with ECMWF studies of SHEBA data by Beesley et al. (2000).

Overall, the ECMWF model had relatively few significant errors, though many of these were caused by similar problems found in other models. The ECMWF not only had difficulty tracking the first two disturbances of synoptic regime I, but also overestimated their intensity by up to 10 hPa. Like the Eta Model, the ECMWF model could not simulate the development and westward motion of secondary disturbances along the coast for forecasts longer than 24 h.

Figure 8 shows that the ECMWF model also had a cold bias over the elevated interior that was directly responsible for the cold temperature and northeast wind biases found near the surface at Toolik Lake throughout MPACE. The model also had a pronounced warm bias over the coastal waters along the NSA and the Beaufort Sea that may have contributed to the significant moist biases found near the surface at Atqasuk during MPACE (Fig. 7). During synoptic regime III, the model slightly overestimated the pressure gradient across the NSA, leading to a midlevel southerly bias at Barrow and Oliktok Point. The tracks of the system and its intensity were simulated very well.

Fig. 8.

Difference plot comparing the 1200 UTC 6 October analysis and 1200 UTC 3 Oct 72-h forecasts for ECMWF surface temperatures. Contours are plotted in K with positive (warm) biases denoted by solid lines and negative (cold) biases denoted by dashed lines.

Fig. 8.

Difference plot comparing the 1200 UTC 6 October analysis and 1200 UTC 3 Oct 72-h forecasts for ECMWF surface temperatures. Contours are plotted in K with positive (warm) biases denoted by solid lines and negative (cold) biases denoted by dashed lines.

c. GMAO model performance

The GMAO model produced temperature, moisture, and wind forecasts for the four MPACE sites. While forecasts for the GMAO were made out to 120 h in 12-h increments, the first forecast is issued at 24, not 12, h. Like the Eta Model, the GMAO model had some variables where both significant positive and negative biases were found and are referred to as varied biases in the figures. Temperature forecasts had the largest errors during synoptic regime III at 1000 and 975 hPa when significant cold biases of 3 K were found at Barrow, Atqasuk, and Oliktok Point (Fig. 9). Significant cold biases up to 2.5 K were found during synoptic regime II at Oliktok (1000–975 hPa) and Toolik Lake (925–900 hPa). The mean absolute errors were identical to the mean bias errors and the standard deviations were below 2.5 K, illustrating a clear, consistent cold bias near the surface. For relative humidity, the model had a moist bias of 10%–50% at all four sites in the 550–200-hPa layer throughout MPACE. Mean absolute errors generally were within 5% of the mean bias errors in this layer, showing that the moist bias was consistent. At altitudes below 700 hPa both dry and moist biases of less than 20% were recorded at all sites for the MPACE period, with the exception of a 40% moist bias at Toolik Lake from 925 to 500 hPa during synoptic regime III. Overall, the errors were slightly larger in synoptic regime III with moist biases of up to 60% at Atqasuk.

Fig. 9.

Statistically significant (80% CI) temperature, relative humidity, and u- and υ-wind component errors for the GMAO model at the four MPACE sites. The horizontal widths of the error bars are proportional to the magnitudes of the errors with sample bars given on the right to assist the reader. Overlapping bars are also given black borders.

Fig. 9.

Statistically significant (80% CI) temperature, relative humidity, and u- and υ-wind component errors for the GMAO model at the four MPACE sites. The horizontal widths of the error bars are proportional to the magnitudes of the errors with sample bars given on the right to assist the reader. Overlapping bars are also given black borders.

Figure 9 shows significant GMAO model wind errors at the four MPACE sites. The majority of the u-wind errors were found at Oliktok Point where westerly biases of 2–3 m s−1 can be found from 1000 to 700 hPa, and varied biases of 2 m s−1 from 700 to 50 hPa. Westerly biases of 2–4 m s−1 were scattered throughout the vertical column at Oliktok Point, while easterly biases of 3–4 m s−1 were found at Toolik Lake at 925 and 250–50 hPa during synoptic regime II. More substantial errors were recorded in the υ-wind component with significant southerly biases of 5–10 m s−1 from 850 to 250 hPa being common at all sites during synoptic regime III. Large errors found in the 500–200-hPa layer during synoptic regime III indicate a misrepresented or misplaced jet stream. The GMAO model’s wind performance was much worse at Oliktok Point relative to the other MPACE sites.

Overall, the GMAO underperformed relative to the other models, with the most substantial wind and moisture errors. The GMAO did not forecast the tracks of the first two systems well outside of 24 forecast hours during synoptic regime I and overestimated their intensity more than any other model with errors in mean sea level pressure of up to 18 hPa. Throughout MPACE, the model areal extents of cloud shields in extratropical lows were too large, particularly in the occluded sections of decaying lows. This can be attributed to the moist bias found at all sites in the mid- to upper troposphere where biases as high as 60% were found. The GMAO model overestimated the intensity of the explosive low that dominated synoptic regime III and also decayed much too slowly in forecasts, contributing to a strong southwesterly wind bias and accompanying warm and moist bias over the NSA during that regime. Unlike the other models, the GMAO also had trouble tracking the system, placing it farther north than it actually was. This further contributed to the westerly wind bias during this time period.

d. RAMS model performance

The RAMS model predicted temperatures, relative humidities, winds, radiation, and cloud fields across the NSA out to 48 forecast hours. Since the domain associated with the fine-resolution grid (4 km) was small, Toolik Lake was not contained in the domain and is not included in the comparisons. Figure 10 shows RAMS mean temperature errors at Barrow, Atqasuk, and Oliktok Point from 0 to 48 h. Statistically significant warm biases of 3–4 K at Barrow were common in the 300–250-hPa layer from 12 to 48 h. Mean absolute errors were generally within 0.5 K of the mean errors, showing that temperature biases were consistent. During synoptic regime III at 1000 hPa, RAMS had warm biases of 5 K at Barrow and 4 K cold biases at Oliktok Point. Forecasting relative humidity proved a more difficult task for the RAMS model. Significant moist biases of 20%–40% appear at all three sites in the 700–250-hPa layer for the MPACE period and synoptic regime III. Mean absolute errors were within about 3% of the mean errors, illustrating the consistency of this bias.

Fig. 10.

Statistically significant (80% CI) temperature, relative humidity, and u- and υ-wind component errors for the RAMS model at three MPACE sites. The horizontal widths of the error bars are proportional to the magnitudes of the errors with sample bars given on the right to assist the reader.

Fig. 10.

Statistically significant (80% CI) temperature, relative humidity, and u- and υ-wind component errors for the RAMS model at three MPACE sites. The horizontal widths of the error bars are proportional to the magnitudes of the errors with sample bars given on the right to assist the reader.

RAMS wind field forecasts (Fig. 10) showed no significant errors that spanned the entire MPACE period. During synoptic regime III the model had a 4–10 m s−1 easterly bias in the 1000–800- and 500–475-hPa layers at Oliktok Point along with a 4–7 m s−1 southerly bias at Barrow in the 525–500- and 300–275-hPa layers. Smaller errors of up to 3 m s−1 were found scattered at Atqasuk and Oliktok Point during synoptic regime II. Errors during synoptic regime III at Oliktok Point were typically about double the magnitude observed at the other sites and, overall, the RAMS model’s performance was inferior at Oliktok Point compared to Barrow and Atqasuk.

Figure 11 shows a time series of the RAMS average forecast cloud fraction, downwelling shortwave radiation, and downwelling longwave radiation at Barrow along with observed values during MPACE. Figure 12 shows the RAMS average forecasts of downwelling short- and longwave radiation versus observations together with errors in the model radiation versus errors in the model cloud fractions at Barrow. Overall, RAMS incident shortwave irradiances were better than those for the Eta Model, with a slight but significant positive bias under 10 W m−2. The model longwave radiation forecasts were also accurate, though a negative bias is apparent. A closer inspection revealed that the model underestimated incident longwave radiation, with significant errors of up to 50 W m−2 in short-term forecasts decreasing steadily to less than 10 W m−2 as time progressed out to 48 forecast hours. Errors smaller than 20 W m−2 in magnitude were not statistically significant. Total radiation errors followed the same general trends as the longwave radiation with a slight negative bias prevailing. A closer look showed the same trend within the data as the longwave radiation forecasts with mean bias errors of 30 and 45 W m−2 at Barrow and Atqasuk, decreasing to around 5 and 15 W m−2 as time progressed from 6 to 48 forecast hours. Once again, errors larger than 20 W m−2 in magnitude were statistically significant. These results were not surprising given that the model initializes cloud and radiation fields to zero for the analyses and prognoses them based on other variables in future time steps. As a result, some time is needed for cloud fields to become realistic and the 0-h forecasts were not included in the comparisons for either of these variables.

Fig. 11.

Time series of RAMS average forecast cloud fraction, downwelling shortwave radiation, and downwelling longwave radiation along with observed values at Barrow, during MPACE.

Fig. 11.

Time series of RAMS average forecast cloud fraction, downwelling shortwave radiation, and downwelling longwave radiation along with observed values at Barrow, during MPACE.

Fig. 12.

(left) RAMS average forecast downwelling shortwave and longwave radiation vs observed values at Barrow, during MPACE. (right) RAMS shortwave and longwave radiation errors (relative to observations) vs RAMS cloud fraction errors (also relative to observations) at Barrow, during MPACE.

Fig. 12.

(left) RAMS average forecast downwelling shortwave and longwave radiation vs observed values at Barrow, during MPACE. (right) RAMS shortwave and longwave radiation errors (relative to observations) vs RAMS cloud fraction errors (also relative to observations) at Barrow, during MPACE.

For cloud fields, the RAMS underpredicted cloud cover by 9.5% during MPACE, on par with the Eta Model (Fig. 11). For synoptic regimes I–III the model underpredicted cloud cover by 18.5%, 4.5%, and 16.1%, respectively, showing that the RAMS model did an excellent job during synoptic regime II in forecasting boundary layer stratus clouds. The overall differences during MPACE and during synoptic regime I were statistically significant. Like the Eta Model, it is intriguing to see that RAMS underestimates longwave radiation and cloud cover fraction while overestimating moisture content throughout virtually the entire troposphere. This may indicate a problem with subgrid cloud parameterizations in these models. Figure 12 shows RAMS radiation errors versus RAMS cloud fraction errors during MPACE. A correlation exists between longwave radiation and cloud fraction errors, as most of the data are clustered in the lower left (indicating negative errors for both fields) and upper right (indicating positive errors for both fields) quadrants. Shortwave radiation correlations were less conclusive.

The RAMS model performed reasonably well relative to the other models. Specifically, the small-scale horizontal grid likely contributed to the improved performance in handling clouds over the coastal NSA during synoptic regime II. However, with forecasts going out to only 48 h and not encompassing Toolik Lake, it may be that more limited comparisons failed to detect some of the model’s shortcomings that would have been picked up in longer forecasts and by the inclusion of Toolik Lake. The RAMS model vertical grid, which is somewhat coarse for a mesoscale model, also influenced the results, though it is unclear how changing the resolution would have affected its performance.

Like the other models, the RAMS model also had trouble tracking the disturbances in synoptic regime I. During synoptic regime II, RAMS had a moist bias over the Arctic sea ice along with a warm bias of 2–3 K over coastal NSA waters that contributed to a moist bias at 1000 hPa at Barrow, Atqasuk, and Oliktok Point. During synoptic regime III, the model had a large midtropospheric moist bias that contributed to an overestimation of the cloud shield surrounding the explosive cyclone. The model captured the track of the system well. Throughout MPACE, except for regime III, the RAMS model also had a warm bias over the Brooks Range, contrasting the cold bias found in the other models. These influenced the pressure gradients over the NSA and contributed to some of the wind biases found during MPACE.

The difference in resolution for the RAMS model compared to the other models could be a point of contention, as some readers could be worried that small-scale phenomena might contaminate the results. However, we believe that the methodology allows for meaningful results to be drawn from such a comparison. As explained previously, the data from each model are compared to the observations in an independent manner so that the results from one model comparison do not influence results for any of the other models that are compared. Second, the use of bootstrapping and the length of the study period work to reduce the influence of the occasional outlier. While small-scale phenomena could contribute to both the observations and RAMS output, it is unlikely to influence them in blocks of time over an entire month. This is reinforced by the results for RAMS showing many significant errors that were present throughout the study period. If large numbers of random outliers had been present, there would be no significant errors in the data. Furthermore, many of the RAMS errors, such as the mid- to upper-tropospheric moist biases, were identical to those found in the large-scale models. While some influence from smaller-scale features is inevitable, the overall results from RAMS seem consistent with those from other models and make the results much more credible.

6. Summary and discussion

The Mixed-Phase Arctic Cloud Experiment (MPACE) was conducted from 27 September through 22 October 2004, across the NSA. The fact that 2004 was a high-melt year influenced the synoptic weather conditions by permitting open water along the coastline for a longer period of time than an average year. The synoptic events during MPACE were examined, and there were three distinct synoptic regimes. Synoptic regime I brought unsettled conditions as several disturbances moved through the area. Synoptic regime II was marked by predominantly undisturbed conditions, with high pressure dominating off the pack ice and mixed-phase boundary layer clouds present on nearly ever day during the period. Synoptic regime III was marked by the presence of an intense storm system that brought anomalously warm conditions to the NSA.

Observations during MPACE were compared to output from the Eta, ECMWF, GMAO, and RAMS models using the block bootstrap method outlined in Wilks (1997) and Marchand et al. (2006). Overall, the bootstrapping results generally reinforced findings from conventional statistical comparisons. Model outputs with the largest mean bias errors were generally the ones that were captured and declared statistically significant in the bootstrapping process. As one would expect with a 80% confidence interval, there were a few cases with large mean bias errors that were not significant because error standard deviations were high, indicating a large spread in the data.

Temperature forecasts for all models were good at the MPACE sites, and models had the fewest significant errors predicting temperature. Forecasting moisture and wind proved to be more difficult for the models, especially aloft in the 500–300-hPa layer. The GMAO and RAMS models had moist errors as high as 40% or more in this layer, while the Eta Model had errors of 10%–20%. The GMAO model also had the largest wind errors in this layer with significant southerly errors of 10 m s−1 or more. The ECMWF model had only a handful of significant moisture and wind errors.

Both the Eta and RAMS models significantly overpredicted shortwave radiation and underpredicted longwave radiation. Given these findings, it was not unexpected to see the Eta and RAMS models underestimate cloud cover fraction as well. What was surprising was the fact that both models underestimated cloud cover despite having significant moist biases in the mid- to upper troposphere. Both the Eta and RAMS models underestimated cloud cover at Barrow by about 10% overall. However, the RAMS errors were concentrated to a few periods when breaks in the low-level stratus were forecasted, but did not occur, while the Eta Model had more consistent, but smaller, errors.

There were several reasons why the models had the significant errors discussed above and in section 5, with each regime posing its own set of problems. The first two systems that influenced the NSA during synoptic regime I had tracks that proved difficult for the models to predict. These disturbances moved from the Aleutian Islands into central Alaska, turned due north toward the Arctic Ocean, then spawned a secondary cyclone in the Beaufort Sea that moved westward for sometime before stalling and moving back eastward. While this behavior was previously observed by Wylie (1999) during SHEBA for both anticyclones and cyclones, none of the models could accurately forecast these features beyond 24 forecast hours. As forecast times increased beyond that point, accuracy decreased markedly. By 72 forecast hours, not only was the forecast track of the parent cyclone considerably worse, but the models did not forecast the development of a secondary cyclone, missing the system entirely. Errors in forecasting the track of the disturbances associated with the second short wave in synoptic regime I are highlighted in Figs. 13 and 14, which show the track of each storm based on mean sea level pressure. In addition to the errors in forecasting the tracks of these storms, each of the models also mishandled the intensities of the storms, magnifying the errors further.

Fig. 13.

Plot of tracks for disturbances associated with the second short wave of synoptic regime I. Two datasets showing the primary low (black) and the secondary low (gray) are shown. For each dataset the analysis (solid) and 24-h forecast tracks for the ECMWF (long dashed), Eta (short dashed), and GMAO (dotted–dashed) models are shown.

Fig. 13.

Plot of tracks for disturbances associated with the second short wave of synoptic regime I. Two datasets showing the primary low (black) and the secondary low (gray) are shown. For each dataset the analysis (solid) and 24-h forecast tracks for the ECMWF (long dashed), Eta (short dashed), and GMAO (dotted–dashed) models are shown.

Fig. 14.

Plot of the track for the primary low associated with the second short wave in synoptic regime I. The analysis (solid) and 72-h forecast tracks for the ECMWF (long dashed), Eta (short dashed), and GMAO (dotted–dashed) models are shown.

Fig. 14.

Plot of the track for the primary low associated with the second short wave in synoptic regime I. The analysis (solid) and 72-h forecast tracks for the ECMWF (long dashed), Eta (short dashed), and GMAO (dotted–dashed) models are shown.

The second synoptic regime produced fewer and smaller biases among the models than those found during synoptic regime III. However, given the conditions during this period, they are no less important. If a model cannot correct itself over a period of over a week with almost identical conditions, this may be an indicator of biases that are caused by the model parameterizations and not by external factors such as missing tracks of storm systems. Significant biases found during regime II, such as large mid- to upper-tropospheric moist biases and southerly wind biases in the models, turned out to be among the most frequent errors in the study. Since the models placed the high pressure system well, and there were no large disturbances during this time, it is difficult to pinpoint the cause of all the errors here. Possible contributors for errors near the surface include a warm bias over the Arctic Ocean along the coastline of the NSA and warm or cold biases over the elevated interior, which in turn are most likely a reflection of errors within the model physical parameterizations.

Errors in the third synoptic regime were tied to how the models handled the explosive cyclone that developed. Given the magnitudes of some of the errors during this period, the track of this storm was forecasted surprisingly well by the models. What caused the significant errors were difficulties simulating the intensity, fast development, and subsequent decay of the cyclone. This led to large wind errors that carried miscalculations over into other variables.

Acknowledgments

We thank all of the dedicated volunteers who helped in the collection of MPACE data during the fall 2004 season. We owe a special thanks to Mark Roulston for his input on the statistical comparisons. We obtained these data through the U.S. Department of Energy as part of the Atmospheric Radiation Measurement Program Climate Research Facility. We thank Alexander Avramov, Chad Bahrmann, Renate Hagedorn, and Mahendra Karki for their help in acquiring the model outputs from the RAMS model, the NCEP Eta Model, the ECMWF model, and the GMAO model, respectively. This research was supported by the Office of Biological and Environmental Research of the U.S. Department of Energy (DE-FG02-05ER64058) as part of the Atmospheric Radiation Measurement Program.

REFERENCES

REFERENCES
Allen
,
D. J.
,
P.
Kasibhatla
,
A. M.
Thompson
,
R. B.
Rood
,
B. G.
Doddridge
,
K. E.
Pickering
,
R. D.
Hudson
, and
S-J.
Lin
,
1996
:
Transport induced interannual variability of carbon monoxide determined using a chemistry and transport model.
J. Geophys. Res.
,
101
,
28655
28669
.
Beesley
,
J. A.
,
C. S.
Bretherton
,
C.
Jakob
,
E. L.
Andreas
,
J. M.
Intrieri
, and
T. A.
Uttal
,
2000
:
A comparison of cloud and boundary layer variables in the ECMWF forecast model with observations at Surface Heat Budget of the Arctic Ocean (SHEBA) ice camp.
J. Geophys. Res.
,
105
,
12337
12349
.
Betts
,
A. K.
,
F.
Chen
,
K. E.
Mitchell
, and
Z. I.
Jancic
,
1997
:
Assessment of the land surface and boundary layer models in two operational versions of the NCEP Eta Model using FIFE data.
Mon. Wea. Rev.
,
125
,
2896
2916
.
Black
,
T. L.
,
1994
:
The new NMC mesoscale Eta Model: Description and forecast experiments.
Wea. Forecasting
,
9
,
265
278
.
Black
,
T. L.
, and
Coauthors
,
1997
:
Changes to the Eta forecast systems.
NWS Tech. Procedures Bull. 441, 16 pp
.
Bromwich
,
D. H.
, and
S. H.
Wang
,
2005
:
Evaluation of the NCEP–NCAR and ECMWF 15- and 40-yr reanalyses using rawinsonde data from two independent Arctic field experiments.
Mon. Wea. Rev.
,
133
,
3562
3578
.
Bullock
,
T. A.
, and
J. R.
Gyakum
,
1993
:
A diagnostic study of cyclogenesis in the western North Pacific Ocean.
Mon. Wea. Rev.
,
121
,
65
75
.
Cotton
,
W. R.
, and
Coauthors
,
2003
:
RAMS 2001: Current status and future directions.
Meteor. Atmos. Phys.
,
82
,
5
29
.
Curry
,
J. A.
, and
G. F.
Herman
,
1985a
:
Infrared radiative properties of summertime Arctic stratus clouds.
J. Appl. Meteor.
,
24
,
525
538
.
Curry
,
J. A.
, and
G. F.
Herman
,
1985b
:
Relationships between large-scale heat and moisture budgets and the occurence of Arctic stratus clouds.
Mon. Wea. Rev.
,
113
,
1441
1457
.
Curry
,
J. A.
, and
Coauthors
,
2000
:
FIRE Arctic Clouds Experiment.
Bull. Amer. Meteor. Soc.
,
81
,
5
29
.
ECMWF
,
cited
.
2007a
:
ECMWF technical information: model level definitions.
.
ECMWF
,
cited
.
2007b
:
Full scientific documentation of the ECMWF Integrated Forecast System (IFS).
.
ECMWF
,
cited
.
2008a
:
ECMWF overview.
.
ECMWF
,
cited
.
2008b
:
The evolution of the ECMWF analysis and forecasting system.
.
Efron
,
B.
, and
R.
Tibshirani
,
1991
:
Statistical data analysis in the computer age.
Science
,
253
,
390
395
.
Eloranta
,
E. W.
,
2005
:
Lidar: Range-Resolved Optical Remote Sensing of the Atmosphere.
Springer-Verlag, 460 pp
.
Fels
,
S. B.
, and
M. D.
Schwarzkopf
,
1975
:
The simplified exchange approximation: A new method for radiative transfer calculations.
J. Atmos. Sci.
,
32
,
1475
1488
.
Fett
,
R. W.
,
S. D.
Burk
,
W. T.
Thompson
, and
T. L.
Kozo
,
1994
:
Environmental phenomena of the Beaufort Sea observed during the Leads Experiment.
Bull. Amer. Meteor. Soc.
,
75
,
2131
2145
.
Gyakum
,
J. R.
,
J. R.
Anderson
,
R. H.
Grumm
, and
E. L.
Gruner
,
1989
:
North Pacific cold-season surface cyclone activity: 1975–1983.
Mon. Wea. Rev.
,
117
,
1141
1155
.
Harrington
,
J. Y.
,
1997
:
The effects of radiative and microphysical processes on simulated warm and transition season Arctic stratus. Ph.D. dissertation, Atmospheric Science Paper 637, Dept. of Atmospheric Science, Colorado State University, 289 pp
.
Harrington
,
J. Y.
,
T.
Reisin
,
W. R.
Cotton
, and
S. M.
Kreidenweis
,
1999
:
Cloud resolving simulations of Arctic stratus. Part II: Transition-season clouds.
Atmos. Res.
,
55
,
45
75
.
Herman
,
G.
, and
R.
Goody
,
1976
:
Formation and persistence of summertime Arctic stratus clouds.
J. Atmos. Sci.
,
33
,
1537
1553
.
Hong
,
S. Y.
,
H. M.
Yuang
, and
Q.
Zhao
,
1998
:
Implementing of prognostic cloud scheme for a regional spectral model.
Mon. Wea. Rev.
,
126
,
2621
2639
.
Hortal
,
M.
, and
A. J.
Simmons
,
1991
:
Use of reduced Gaussian grids in spectral models.
Mon. Wea. Rev.
,
119
,
1057
1074
.
Keegan
,
T. J.
,
1958
:
Arctic synoptic activity in winter.
J. Meteor.
,
15
,
513
521
.
Key
,
J. R.
, and
A. C.
Chan
,
1999
:
Multidecadal global and regional trends in 1000 mb and 500 mb cyclone frequencies.
Geophys. Res. Lett.
,
26
,
2053
2056
.
Klemp
,
J. B.
, and
R. B.
Wilhelmson
,
1978
:
The simulation of three-dimensional convective storm dynamics.
J. Atmos. Sci.
,
35
,
1070
1096
.
Lacis
,
A. A.
, and
J. E.
Hansen
,
1974
:
A parameterization of the absorption of solar radiation in the earth’s atmosphere.
J. Atmos. Sci.
,
31
,
118
133
.
Lin
,
S. J.
,
2004
:
A vertically Lagrangian finite-volume dynamical core for global models.
Mon. Wea. Rev.
,
132
,
2293
2307
.
Lin
,
S. J.
,
W. C.
Chao
,
Y. C.
Sud
, and
G. K.
Walker
,
1994
:
A class of the van Leer–type transport schemes and its applications to the moisture transport in a general circulation model.
Mon. Wea. Rev.
,
122
,
1575
1593
.
Lindsay
,
R. W.
,
J.
Zhang
,
A.
Schweiger
,
M.
Steele
, and
H.
Stern
,
2009
:
Arctic sea-ice retreat in 2007 follows thinning trend.
J. Climate
,
22
,
165
176
.
Marchand
,
R.
, and
Coauthors
,
2006
:
A bootstrap technique for testing the relationship between local-scale radar observations of cloud occurrence and large-scale atmospheric fields.
J. Atmos. Sci.
,
63
,
2813
2830
.
Mattioli
,
V.
,
E. R.
Westwater
,
D.
Cimini
,
J. C.
Liljegren
,
B. M.
Lesht
,
S. I.
Gutman
, and
F. J.
Schmidlin
,
2007
:
Analysis of radiosonde and ground-based remotely sensed PWV data from the 2004 North Slope of Alaska Arctic Winter Radiometric Experiment.
J. Atmos. Oceanic Technol.
,
24
,
415
431
.
Mesinger
,
F.
,
Z. I.
Janjic
,
S.
Kickovic
,
D.
Gavrilov
, and
D. G.
Deaven
,
1988
:
The step-mountain coordinate: Model description and performance for cases of Alpine lee cyclogenesis and for a case of Appalachian redevelopment.
Mon. Wea. Rev.
,
116
,
1493
1518
.
Meyers
,
M. P.
,
R. L.
Walko
,
J. Y.
Harrington
, and
W. R.
Cotton
,
1997
:
New RAMS cloud microphysics parameterization. Part II: The two-moment scheme.
Atmos. Res.
,
45
,
3
39
.
Norris
,
P. M.
, and
A. M.
da Silva
,
2007
:
Assimilation of satellite cloud data in the GMAO finite-volume data assimilation system using a parameter estimation method. Part I: Motivation and algorithm description.
J. Atmos. Sci.
,
64
,
3880
3895
.
Overland
,
J. E.
, and
C. H.
Pease
,
1982
:
Cyclone climatology of the Bering Sea and its relation to sea ice extent.
Mon. Wea. Rev.
,
110
,
5
13
.
Parkinson
,
C. L.
,
D. J.
Cavalieri
,
P.
Gloersen
,
H. J.
Zwally
, and
J. C.
Comiso
,
1999
:
Arctic sea ice extents, areas, and trends, 1978–1996.
J. Geophys. Res.
,
104
,
20837
20856
.
Partington
,
K.
,
T.
Flynn
,
D.
Lamb
,
C.
Bertoia
, and
K.
Dedrick
,
2003
:
Late twentieth century Northern Hemisphere sea-ice record from U.S. National Ice Center ice charts.
J. Geophys. Res.
,
108
,
3343
.
doi:10.01029/2001JC001603
.
Riphagen
,
H. A.
,
C. L.
Bruyere
,
W.
Jordaan
,
E. R.
Poolman
, and
J. D.
GertenBach
,
2002
:
Experiments with the NCEP regional Eta Model at the South African Weather Bureau, with emphasis on terrain representation and its effect on precipitation predictions.
Mon. Wea. Rev.
,
130
,
1246
1263
.
Rogers
,
E.
,
T. L.
Black
,
D. G.
Deaven
,
G. J.
DiMego
,
Q.
Zhao
,
M.
Baldwin
,
N. W.
Junker
, and
Y.
Lin
,
1996
:
Changes to the “early” Eta analysis/forecast system at the National Centers for Environmental Prediction.
Wea. Forecasting
,
11
,
391
413
.
Rood
,
R. B.
,
1987
:
Numerical advection algorithms and their role in atmospheric transport and chemistry models.
Rev. Geophys.
,
25
,
71
100
.
Rothrock
,
D. A.
,
J.
Zhang
, and
Y.
Yu
,
2003
:
The Arctic ice thickness anomaly of the 1990s: A consistent view from observations and models.
J. Geophys. Res.
,
108
,
3083
.
doi:10.1029/2001JC001208
.
Sanders
,
F.
, and
J. R.
Gyakum
,
1980
:
Synoptic-dynamic climatology of the “bomb”.
Mon. Wea. Rev.
,
108
,
1589
1606
.
Sassen
,
K.
,
1994
:
Advances in polarization diversity lidar for cloud remote sensing.
Proc. IEEE Remote Sens. Instruments Environ. Res.
,
82
,
1907
1914
.
Serreze
,
M. C.
, and
R. G.
Barry
,
1988
:
Synoptic activity in the Arctic basin.
J. Climate
,
1
,
1276
1295
.
Serreze
,
M. C.
,
J. E.
Box
,
R. G.
Barry
, and
J. E.
Walsh
,
1993
:
Characteristics of Arctic synoptic activity, 1952–1989.
Meteor. Atmos. Phys.
,
51
,
147
164
.
Serreze
,
M. C.
, and
Coauthors
,
2003
:
A record minimum Arctic sea ice extent and area in 2002.
Geophys. Res. Lett.
,
30
,
1110
.
doi:10.1029/2002GL016406
.
Simmons
,
A. J.
, and
D. M.
Burridge
,
1981
:
An energy and angular-momentum conserving vertical finite-difference scheme and hybrid vertical coordinates.
Mon. Wea. Rev.
,
109
,
758
766
.
Slingo
,
J. M.
,
1987
:
The development of a cloud prediction model for the ECMWF model.
Quart. J. Roy. Meteor. Soc.
,
113
,
899
927
.
Steffen
,
K.
, and
T.
DeMaria
,
1996
:
Surface energy fluxes of Arctic winter sea ice in the Barrow Strait.
J. Appl. Meteor.
,
35
,
2067
2079
.
Tarasova
,
T. A.
,
J. P.
Fernandez
,
I. A.
Pisnichenko
,
J. A.
Marengo
,
J. C.
Ceballos
, and
M. J.
Bottino
,
2006
:
Impact of new solar radiation parameterization in the Eta Model on the simulation of summer climate over South America.
J. Appl. Meteor. Climatol.
,
45
,
318
333
.
Uttal
,
T.
, and
Coauthors
,
2002
:
Surface heat budget of the Arctic Ocean.
Bull. Amer. Meteor. Soc.
,
83
,
255
275
.
Verlinde
,
J.
, and
Coauthors
,
2007
:
The Mixed-Phase Arctic Cloud Experiment.
Bull. Amer. Meteor. Soc.
,
88
,
205
221
.
Wilks
,
D. S.
,
1997
:
Resampling hypothesis tests for autocorrelated fields.
J. Climate
,
10
,
65
82
.
Wylie
,
D. P.
,
1999
:
Arctic weather during the FIRE/ACE flights in 1998.
J. Geophys. Res.
,
106
,
15363
15375
.
Xu
,
K. M.
, and
D. A.
Randall
,
1996
:
A semiempirical cloudiness parameterization for use in climate models.
J. Atmos. Sci.
,
53
,
3084
3102
.
Yarnal
,
B.
, and
K. G.
Henderson
,
1989
:
A climatology of polar low cyclogenetic regions over the North Pacific Ocean.
J. Climate
,
2
,
1476
1491
.
Yucel
,
I.
,
W. J.
Shuttleworth
,
J.
Washburne
, and
F.
Chen
,
1998
:
Evaluating NCEP Eta Model–derived data against observations.
Mon. Wea. Rev.
,
126
,
1977
1991
.
Zhang
,
X.
,
J. E.
Walsh
,
J.
Zhang
,
U. S.
Bhatt
, and
M.
Ikeda
,
2004
:
Climatology and interannual variability of Arctic cyclone activity: 1948–2002.
J. Climate
,
17
,
2300
2317
.

Footnotes

Corresponding author address: Victor Yannuzzi, 3780 30th Ave. South, Apt. 303, Grand Forks, ND 58201. Email: vicyannuzzi@gmail.com