1. Introduction

Comprehensive air–sea interaction and climate studies have been restricted in the past due to the lack of uniformly gridded upper-ocean temperature datasets with a temporal coverage of two to three decades. Unlike meteorological studies, oceanographic studies do not have the luxury of regular time series observations spanning decades. Most historical oceanographic data have been collected from ships, which not only include research vessels but also voluntary observing ships (VOSs). Consequently, due to variations in cruise times and shipping routes, oceanographic data collected from ships provide irregularly distributed information in space and time. Hence, a major consideration when analyzing historical ocean data in ocean–atmosphere interaction and other large-scale climate studies is the distribution of these data in space and time.

Given the recent increase in the number of ocean temperature profiles now available through present-day ocean observing systems—for example, through the intensive period of expendable bathythermograph (XBT) deployments during the Tropical Ocean Global Atmosphere (TOGA) program—generating uniformly gridded maps of ocean fields on a global scale at sufficient spatial (in three dimensions) and temporal resolution is becoming more computationally expensive, even for present-day computers. Nevertheless, the demands of present-day climate research are such that scientists require large- to global-scale gridded fields, with objective measures of the observational/mapping error fields being essential, in order to quantify estimates of climate variability and climate change. It is these data that will be used to validate and verify the predictions from climate change scenarios. The limiting factor in producing gridded maps, however, remains the large computational costs involved with the objective mapping procedures (e.g., Gandin 1963; Bretherton et al. 1976; Roemmich 1983).

Fukumori and Wunsch (1991) have more recently explored the use of empirical orthogonal function (EOF) analysis, by the singular value decomposition (SVD), to reduce the volume of data associated with profiles of temperature, salinity, oxygen, and nutrients obtained along a series of sections across the North Atlantic basin between 1981 and 1985. They showed that these data can be decomposed into a collection of EOFs, and that the number of significant modes required to represent the vertical (and horizontal) structure in the data may be dramatically reduced as a consequence of the inherent vertical and horizontal correlations within the ocean. A similar analysis was applied successfully by Bindoff and Wunsch (1992) to the South Pacific Ocean.

As indicated in their review, Fukumori and Wunsch’s (1991) application of EOFs to hydrography was, at the time, in itself not new [prior work, for example, includes the more locally/regionally focused study of Frankignoul (1981)]. Rather, Fukumori and Wunsch (1991) applied the methodology to the entire North Atlantic basin using both hydrographic and chemical observations. In the present paper, we demonstrate, by application to historical mechanical bathythermograph (MBT) and XBT observations collected in the southwest Pacific Ocean between 1955 and 1988, that the EOF methodology may be successfully extended to four dimensions through the inclusion of temporal variations in the ocean data, and that the objective mapping of only a few significant modes can produce an unbiased and statistically consistent gridded time history of the large-scale temperature changes with objective error estimates.

The combined technique described here adds to the previous literature primarily through its (i) improved estimate of the first-guess (a priori) mean field as a polynomial in two-dimensional space with annual and semiannual harmonics in time; and (ii) addition of time to produce four-dimensional gridded maps in polar coordinates, T(λ, ϕ, z, t), directly.

Other major strengths of the method are the (i) production of smooth, gridded maps in both space and time;(ii) identification and quantification of noise in the data;(iii) preservation of the vertical structure in the data; (iv) retention of only the most important information in the data, and as a consequence; (v) vastly improved efficiency in computing time.

This paper describes the data (section 2), the methods used to map the data (section 3), and the results from the technique applied to the MBT and XBT data, together with an analysis of the residuals and mapped fields (section 4). A discussion of the results and some concluding remarks are presented in section 5.

2. Data

The ocean temperature data used in this study are from the National Oceanographic Data Center, Washington, DC, (National Oceanographic Data Center 1991) CD-ROM. Most of the available MBT and XBT open ocean data, about 40 000 casts, were examined for the southwest Pacific Ocean region 0°–50°S, 140°E–180° covering the period between 1955 and 1988. These data were selected from the total of 56 000 casts that were available. There are a disproportionately large number of observations along the East Australian shelf off the New South Wales coast, with 30% of the total dataset located between 30° and 37°S and between the East Australian coast and 154°E.

The application of the technique described in this paper focuses on the climate of the large-scale ocean temperature field, in particular, the broad-scale, upper-ocean temperature variability within the interior of the southwest Pacific. This paper is not concerned with the detailed description and mapping of the temperature variations in the boundary flow of the East Australian Current (EAC). This more complex region is dominated by the seasonal cycle of EAC eddies that pinch off each year (Mulhearn et al. 1986, 1988; Ridgway and Godfrey 1997), making even the time-averaged mean flow difficult to determine on the timescales and space scales of this study. Rather, the main objectives of the ocean climate study (Holbrook and Bindoff 1997, 1999) have been to concentrate on the large-scale features of the subtropical gyre. For the present application of this technique, and given our interests only in the large-scale field, the coastal data were excluded and are not analyzed.

The MBT data were provided at discrete and regular vertical intervals. In contrast, and for data storage efficiency, most of the XBT data were provided at “inflection point” depths. These depths were chosen such that the original set of temperature data could be reproduced by linear interpolation with an error of no more than ±0.10°C (D. Collins of NODC User Services, 1991, personal communication). However, the XBT data that were originally collected and stored at selected levels (or standard depths) were also provided at those selected levels by the NODC. Details and references regarding the accuracy of both instruments are contained in Holbrook (1994).

The temperature data were first linearly interpolated in the vertical onto a 5-m grid between the surface and the maximum depth of 450 m. Quality checks were made to identify any gross errors in the data such as obvious positional errors and outlying values. Figure 1 shows the spatial positions and time distribution of all MBT and XBT casts retained in the data after quality control. Given the large number of observations and the limitations of available computer resources, it was necessary to carry out the data analysis on subsets of the original dataset. Consequently, the study region was divided into four subregions chosen with some reference to the local dynamics Subregion 1 includes the western Coral Sea, Solomon Sea, and the Bismarck Sea [TOGA Coupled Ocean–Atmosphere Response Experiment (COARE) domain]; subregion 2 encompasses the broader tropical southwest Pacific basin; subregion 3 extends from the subtropics through to the midlatitudes and defines the western Tasman Sea and is a region of high mesoscale variability in the vicinity of the EAC; and finally subregion 4, a less variable region, mainly represents the eastern Tasman Sea, which is broadly separated from the west by the Lord Howe rise. These subregions contain a 2° overlap at each of the east–west and north–south boundaries in order to minimize sharp edges and discontinuities in the final mapped fields.

The MBT and XBT datasets complement each other well, providing an evenly distributed temporal coverage during the selected period. The main spatial variations in coverage across the study region have resulted mostly from changes in shipping routes and the establishment of the VOS network. Although the MBT data are more sparse than the XBTs in some regions, notably region 2 (cf. Figs. 1a,b), they are also the most comprehensive dataset in the region south of New Zealand due particularly to the intensive period of Southern Ocean cruises between 1956 and 1962. The work by, for example, Wyrtki (1975, 1985) during the 1970s and 1980s and, in general, the renewed interest in El Niño since the late 1960s, have resulted in a more focused program of XBT data collection in the tropical regions since that time. The more central latitudes within the region (15°–40°S) are well covered in space and time over the entire period. It should be noted that the MBT data are shallower and typically extend to 135-m depth, while the XBT data usually extend to 450-m depth or more in the open ocean.

Although the observational coverage is relatively uniform, the density of the data in time and space is sparse. As an annual average, the coverage translates to almost 100 casts per 2° latitude–longitude grid cell (about 25 casts per 1° box). As a monthly climatology (long-term averaged), this data density reduces to about eight casts per 2° cell (about 2 casts per 1° cell). However, at the seasonal scale (3-month increments) for each year over the full 34 years between 1955 and 1988, there is a little less than one observation per 2° spatial cell, with the higher density XBT observation period between 1973 and 1988, averaging just over one observation per 2° cell.

The data density greatly affects the temporal and spatial resolution that can be achieved from these data for this region. As a consequence of these limitations, we have been both realistic and pragmatic with our choices of space scales and timescales used for mapping the temperature field. These choices are based largely on a combination of the (i) observed anisotropy of upper ocean temperatures, with longer zonal scales than meridional scales apparent in the upper ocean (Fig. 2); (ii) consideration of the trade-offs in data resolution between the selection of large length scales for statistical reliability as opposed to the finer spatial and temporal scales; and (iii) spatial and temporal scales in the tropical–subtropical zone for the South Pacific reported in the literature (Meyers et al. 1991).

3. Treatment of the data

b. Prior statistics

Before calculating the EOFs, it is important to obtain prior estimates of both the signal and the background noise in the temperature data. The signal is represented as the difference between the point observations of temperature and the a priori mean temperature at each of the observation points. The a priori noise in the data is defined here as the small-scale variability associated with the unresolved ocean processes, such as internal waves and mesoscale eddies, or errors associated with instrument type and/or reporting technique. The estimate of the a priori noise is the same for both stages II and III since it is an objective estimate of the noise.

1) A priori noise

The variance of the data about its mean vertical structure contains both the signal variance and the noise variance and is not an appropriate measure for weighting the data used in the EOFs (as is commonly done). The noise variance at each depth should instead be estimated independently of the signal variance. In the present technique, it is found from the difference between each temperature value and those for close neighboring casts. Neighboring is referred to here as the shorter spatial and temporal scales that tend to separate individual casts along a cruise track by less than 100 km and less than 1 day. The short timescale here ensures that the selected casts represent those deployed from the same ship, or a separate ship close in time (and space). This scheme overcomes the problem of sampling across different seasons and ensures that noise is represented by mesoscale processes, rather than the large-scale, longer period signals that we wish to resolve.

We have assumed that the a priori noise estimate is homogeneous over the period of the measurements within each subregion. On a year-by-year basis, the oceanographic noise within each region is difficult to estimate and is not reliable, given the data sparsity. The subdivision of the study region spatially, however, permits a simple estimate of the spatial variation in the noise statistics across the southwest Pacific.

Let T_r be the observed temperature for cast r at a selected depth, with signal, s_r, and noise, n_r. Therefore, T_r = s_r + n_r. The mean-square difference between this temperature value and that of a neighboring cast, T_s, at the same depth is

T_rT_s²s_rs_s²n²(1)

where the angle brackets 〈 〉 here denote the expectation operator. The above relation assumes that the signal and noise are uncorrelated, that is, 〈sn〉 = 0 and 〈n_rn_s〉 = 0. Assuming the signal has a much greater correlation scale than the distance between neighboring casts, the first term in Eq. (1) is small compared with the second term and can be ignored. Thus, the a priori noise at depth i becomes

View Expanded

where (T_r − T_s)_k is the kth difference between pairs of distinct casts separated spatially by less than 100 km and temporally by less than 1 day.

Direct application of Eq. (2) to the data within the individual subregions is costly in computer time because of the time required to calculate distances between casts. Since the casts of interest in the above calculation uses only neighbors within 100 km and 1 day of each other, the information from individual casts, say, 15° of latitude apart, is clearly redundant. Consequently, after some initial trials, it was found that the computing time could be dramatically reduced by successively making searches for neighboring casts within 3° latitude bands and intervals of 1 year in time, with little overall change to the results.

2) A priori mean field

In stage I (in preparation for stage II), the a priori mean temperature field, T_apriori = f(ϕ, z, t), was modeled as a second-order polynomial in latitude (ϕ) and depth (z), with annual and semiannual harmonics in time. This a priori model of the seasonal temperature cycle is linear in the unknown coefficients to the polynomial and seasonal harmonics (with a total of 45 regression coefficients) and was found using least squares (e.g., Menke 1989). The complete expression for this polynomial is given in the appendix.

The polynomial model of the a priori mean field is not highly sophisticated. Nevertheless, this model broadly fits the large-scale vertical and meridional structure of the ocean together with the seasonality of upper-ocean temperature variations across the region. We have neglected zonal variations in the temperature field, as zonal variations are much weaker in the ocean interior of the subtropical gyre. Statistics associated with the a priori mean field are presented in section 4.

For the time series products (stage III), the a priori mean field was the estimated monthly climatology (the product of stage II).

c. Singular value decomposition

Let A be an M × N matrix for the data such that M represents the number of standard depths and N is the number of stations. Vertical positions are allocated every 5 m from the surface to 450-m depth, making a total of 91 standard depths. Following Fukumori and Wunsch (1991), the temperature data are normalized by subtracting an a priori mean field and dividing by the a priori noise. Hence, each element of the A matrix is described by

View Expanded

where T_ij is the temperature value at depth i, station j, T_ij is the a priori mean at depth i, station j, and the normalization factor, σ_i, is the a priori error (noise) estimate at depth i.

Each of the four subregions in the southwest Pacific contained in excess of 8000 individual casts. Hence, the number of stations in any one subregion was always far greater than the 91 standard depths. In order to minimize the size of the eigenvalue problem, the data matrix was constructed with depth in the row dimension. Hence, each column of the matrix contained a separate MBT/XBT cast corresponding to a geographic location and time. The SVD was performed on the vertical covariance matrix AA^T for the data within each subregion such that

AA^TUΛ²U^T(4)

where the columns, u_i, of U are the vertical eigenvectors. From the definition of the SVD of the data matrix, A (e.g., Menke 1989; chapter 7.6, p. 119); that is,

AUΛV^T(5)

a simple rearrangement, using the orthogonality property of U (i.e., U⁻¹ = U^T), yields

VΛ⁻¹U^TA^T(6)

where the diagonal elements of the M × M matrix, Λ, are the set of eigenvalues and V is an N × M matrix with each column vector, v_i, representing the amplitudes of the associated vertical eigenvector, u_i. Since each column of the data matrix, A, represents a single cast corresponding to a geographic location and time, each column vector, v_i, represents the time and space variation of the associated vertical eigenvector u_i.

The rank, p, of the matrix A is the minimum dimension of M × N and represents the total number of modes, that is, equal to the total number of eigenvectors or eigenvalues, arising from the SVD. Here, we are primarily interested in the number of significant modes, k ⩽ p, which are above the noise level of the data.

SVD is a numerically expensive procedure because it requires ∼N³ operations. However, since the rank of the covariance matrix is bounded by the minimum of M or N, then for data matrices with one dimension much smaller than the other, as in this problem, huge savings in computer time and memory can be achieved by using the vertical covariance matrix. These savings occur because there are only 91 vertical levels, compared with the 8000⁺ profiles contained in each subregion. As a consequence, the computational cost of determining the EOFs is trivial compared to the objective mapping step.

e. Objective mapping

Objective mapping is used to map each of the retained v_i’s onto the chosen regular horizontal grid. The vertical structure at each grid point for each mapped mode is defined by u_i. The objectively mapped v_i values are given by Bretherton et al. (1976) as

View Expanded

where v_i is the ith eigenvector of the horizontal coefficient matrix, is the mean of the coefficients for the ith mode and C_md and C_dd are, respectively, the model–data and data–data covariance matrices.

An anisotropic Gaussian covariance function (including time) was chosen for the analysis. Each element of the data–data covariance matrix, C_idd, is given by

View Expanded

where Δr_xjk, Δr_yjk, Δr_tjk are the distances in longitude (°), latitude (°), and time (days), respectively, between data points j, and k, l_x, l_y, l_t are the corresponding covariance scales, and s_i and e_i are estimates of the signal and noise variances for the ith mode. The signal variance s_i here is simply the covariance of v_i, while the noise variance e_i of v_i is obtained using the same procedure as for the a priori noise. The model–data covariance matrix C_imd is described by an equation almost identical to Eq. (9);the only difference is that the Kronecker delta term is not included because the model covariance is assumed to be free of noise. Here, the identities Δr_xjk, Δr_yjk, Δr_tjk are the distances in longitude (°), latitude (°), and time (days), respectively, between a data and model grid point.

Computationally, the objective mapping procedure is a costly step in the process of producing the gridded climatology and time series. The procedure involves an inversion of the data–data covariance matrix. This is an N³ operation in time and can become unmanageable for N > 1500 points or so on most good workstations. A major advantage of the EOF analysis procedure combined with objective mapping is that it reduces the number of times that mapping is required, while still retaining the vertical correlations present in the data.

Objective mapping relies on a reasonable choice of scales based on both the underlying physics and the data distribution. As mentioned earlier, we have been both realistic and pragmatic with our choices of space scales and timescales for the ocean temperature field. The only comprehensive assessment of space scales and timescales within the region that the authors are aware of is by Meyers et al. (1991) in the tropical–subtropical South Pacific. From their study, they report median scales for the depth of the 20°C isotherm (a proxy for the depth of the thermocline, which approximates the dominant baroclinic mode structure for the subsurface ocean) across the tropical–subtropical Pacific Ocean as 3° in latitude (meridional length scale), 15° in longitude (zonal length scale) and two months in time. For sea surface temperatures, and hence mixed layer temperatures, these scales are larger.

In the present study, horizontal length scales of 4° in longitude and 2° in latitude and a timescale of 90 days were chosen. The larger scale in longitude was chosen due to the zonality of ocean temperatures, while the choice of grid was selected to be consistent with that of available surface forcing data such as the Comprehensive Ocean–Atmsophere Data Set (Woodruff et al. 1987). Given the results of the Meyers et al. (1991) study, we believe that our choices of scales for the interior of the southwest Pacific are reasonable and consistent with the available data.

f. Reconstruction

The eigenvectors, u_i and v_i, are nondimensional vectors while the singular value, λ_i, is a scaling factor, with λ²_i a measure of the normalized variance of the temperature data in A accounted for by mode i. The total variance of A is the sum of all, p, orthogonal modes (Lawson and Hanson 1974)

A²λ²₁λ²₂λ²_p(10)

where the norm ‖ · ‖ is defined as the square root of the sum of squares of all elements in the matrix

View Expanded

and p is the rank. Selecting a smaller subset of modes, say, up to and including λ_k, then Eq. (5) can be written in the reduced matrix form

AU_kΛ_kV^T_k(12)

where the subscript k denotes that only the first k columns of the matrices are retained. An alternative representation of Eq. (12) is

View Expanded

A full reconstruction of the matrix, and hence the set of normalized temperatures, is achieved by summing these individual terms up to the total number of modes, p. However, reconstruction of the normalized temperatures using only a small number of modes, k, that are able to explain a high proportion of the total variance in the data, is clearly more efficient.

Finally, the posteriori error, σ_ijnorm, is reconstructed using the relation

View Expanded

where p is the rank and u_im and e_jm are, respectively, the mth mode eigenvector and estimated error of the mapped coefficients at the jth grid point and at the ith depth [cf. Bretherton et al. (1976)]. Here, however, the posteriori error [Eq. (14)] is generated using only the selected number of k modes and hence this estimate of the error is incomplete since it does not include the error associated with the neglected (p − k) modes. The dimensional error is represented by

σ̂σ_normσ_apriori(15)

The method used to select the retained modes, k, will be described in the following section.

4. Results

This section describes the various statistics arising from the application of the method to the historical upper-ocean temperature observations in the southwest Pacific. The main criteria of success is that the residuals between the monthly climatology or the time series are free of bias at all depth levels, and that the residuals are statistically consistent with the a priori estimate of the noise. Because this method involves multiple steps in the estimation of the final gridded field, there is no formal guarantee that the estimated temperature climatology or time series is optimal in the sense of minimizing the residuals and is an unbiased estimate of the true field.

Other information gleaned from the analysis include (i) estimates of the spatial variability of the signal variance in the data calculated as the mean squared differences between the data and the polynomial mean as well as between the temperature data and the estimated (mode reconstructed) temperature field for 1° latitude (and longitude) bands, (ii) a simple estimate of the spatial variability of the noise provided by a priori estimates for each subregion and at every 5 m in depth, (iii) a description of the vertical structure in the data for each of the four subregions provided by the vertical modes u_i, and (iv) a description of the contribution to the horizontal and temporal variability in the data for each of the four subregions provided by the expansion coefficients, v_i.

5. Discussion and concluding remarks

The purpose of this paper has been to present an objective method for mapping irregularly sampled ocean temperature profiles, such that a full reconstruction of the four-dimensional temperature field (i.e., three spatial dimensions and time) can be made, that is consistent with our knowledge of the spatial and temporal scales, and with a minimum of computational cost. As a case study, the irregularly spaced upper-ocean temperature profiles used in this study were collected from a mixture of merchant vessels and oceanographic cruises with individual projects in mind rather than long-term monitoring as the goal. In order to generate a useful uniformly gridded temperature time series from such data, it is important that an objective method is used because it provides an unbiased estimate of the entire gridded field together with estimates of the errors associated with the original data distribution through the mapping procedure.

Objective mapping (or objective analysis/interpolation), a least squares procedure, was chosen for this analysis. Unfortunately, for very large datasets, such as the dataset discussed in this paper, objective mapping becomes computationally very costly since it requires a matrix inversion of the large data–data covariance matrix, an N³ operation in time. Nevertheless, given the continuous vertical temperature profiles available from MBT and XBT deployments, optimal use of this vertical information is also desired. A feature of the present study is that it was able to take account of the inherent vertical correlations which physically exist within the ocean. This simple fact hightlights the vertical redundancy in the data and is the key to the savings in computing time afforded by the objective mapping of only a few (five) EOFs, rather than the complete set of 91 vertical levels.

It was found that about 90% of the overall temperature variability in the vertical could, in most cases, be explained by the first five vertical modes. Furthermore, it was found that the original data could be reconstructed to within the a priori noise using only the first five vertical modes. Consequently, the gridded temperature fields at each of the 91 vertical levels could be generated using the objective mapping procedure only five times, providing dramatic computational savings. This combination of EOF analysis and objective mapping was used to produce uniformly gridded datasets of the upper-ocean seasonal mean (monthly climatology) temperature field and also the upper-ocean temperature time series between 1955 and 1988 to 100-m depth and between 1973 and 1988 to 450-m depth on a 3-monthly grid (January, April, July, and October). Each of these datasets was produced on a 2° × 2° spatial grid.

With the interest in better understanding climate variability and change, and the important role that the ocean plays in the coupled climate system, there is now perhaps more than ever before a need to be able to quantify the regional to large-scale changes in ocean and atmosphere temperatures and circulation, with estimates of uncertainty. On the large spatial scale and for long time histories, investigations of this kind often require the analysis of huge quantities of data. The technique described in this paper provides an objective, yet pragmatic approach to quantifying time changes, with errors, in these historical observations for very large datasets.

We are currently extending the technique described here to produce an optimal and sequential method, which allows updating of the vertical eigenvectors and posteriori errors. It is expected that this method will be applied in near–real time as required by the Climate Variability and Predictability Programme and modern forecasting techniques.