Measurements of sea surface temperature at the skin interface () made by an Infrared Sea Surface Temperature Autonomous Radiometer (ISAR) have been used for a number of years to validate satellite sea surface temperature (SST), especially high-accuracy observations such as made by the Advanced Along-Track Scanning Radiometer (AATSR). The ISAR instrument accuracy for measuring is ±0.1 K (Donlon et al.), but to satisfy Quality Assurance Framework for Earth Observation (QA4EO) principles and metrological standards (Joint Committee for Guides in Metrology), an uncertainty model is required. To develop the ISAR uncertainty model, all sources of uncertainty in the instrument are analyzed and an uncertainty value is assigned to each component. Finally, the individual uncertainty components are propagated through the ISAR retrieval algorithm to estimate a total uncertainty for each measurement. The resulting ISAR uncertainty model applied to a 12-yr archive of measurements from the Bay of Biscay shows that 77.6% of the data are expected to be within ±0.1 K and a further 17.2% are within 0.2 K.
This paper is concerned with how to make reliable estimates of the measurement uncertainty associated with observations of sea surface temperature (SST; acronyms are also defined in the appendix) made using a shipborne infrared radiometer, in the context of validating satellite-derived global SST products. The uncertainty analysis is developed for a particular instrument design, the Infrared Sea Surface Temperature Autonomous Radiometer (ISAR) (Donlon et al. 2008) and is applied to a 12-yr SST reference dataset acquired during operational deployments of this type of radiometer in the northeast Atlantic Ocean (The data are archived at the NEODC, http://www.ceda.ac.uk/). Shipborne radiometry is already recognized as an appropriate means of delivering the in situ reference measurements of ocean surface skin temperature that are needed for independent validation of global satellite-derived SST datasets (see GCOS 2011, p. 45; Minnett and Corlett 2012). Case studies (Noyes et al. 2006; Wimmer et al. 2012) have demonstrated the effectiveness of using collocated coincident shipborne radiometry to validate SST data products derived from the Advanced Along-Track Scanning Radiometer (AATSR) carried on the European Space Agency (ESA) satellite Environmental Satellite (Envisat). So far, the quality control of the ISAR data used for validation of satellite data has consisted simply of tests to ensure that the measurement uncertainty during a particular ISAR deployment was less than a given target, typically within ±0.1 K. However, when satellite-derived SST datasets are to be used for the compilation of climate data records (CDRs) in which SST is regarded as an essential climate variable (ECV) (Bojinski et al. 2014), then a more rigorous approach to satellite data quality should be followed, as recommended by Barker et al. (2015). Most notably this implies that the in situ SST observations used for validating satellite datasets should be accompanied with estimates of the uncertainty of each SST record, in accordance with formal metrological protocols (JCGM 2008) that are now incorporated into the Committee on Earth Observation Satellites (CEOS) specifications for CDRs (QA4EO Task Team 2010). Until now, published ISAR-measured in situ SST datasets (e.g., Minnett 2011; Wimmer et al. 2012; Guan et al. 2011; Dybkjær et al. 2012) have not met this requirement, which formally could disqualify them from being used to qualify SST CDRs. To remedy this shortcoming, this paper provides a thorough critique of the uncertainties inherent in the operation of an ISAR, and it describes the construction of a robust uncertainty model for incorporation into the SST retrieval component of the ISAR postprocessing software.
In the rest of the paper, we first outline the procedures used by an ISAR to determine the sea surface temperature at the skin interface () using radiometric measurements: this is necessary background knowledge for characterizing the measurement uncertainties. Section 3 explains the rationale for adopting an uncertainty approach for qualifying ISAR data. It then identifies the main sources and types of error and variability in the instrument and its operating environment that introduce uncertainties into the SST data output. Section 4 explores each of these issues separately, aiming for a robust estimate of the amount of uncertainty that each contributes individually to the overall uncertainty of ISAR. In section 5 the different uncertainties are propagated through the ISAR retrieval processor, to allow the overall uncertainty to be reliably estimated, independently for every ISAR record, as required by the Quality Assurance Framework for Earth Observation (QA4EO) principles. The whole archive of ISAR data for the Bay of Biscay transects has been reprocessed to generate uncertainty estimates attached to the records and the results are discussed in section 6. The conclusions in section 7 include consideration of the value added to the ISAR datasets by including uncertainty estimates and the impact this may have on the quality of future satellite SST validation activities.
2. Background-measuring with ISAR
a. measurement basis of the ISAR radiometer
The ISAR instrument shown in Fig. 1 is specifically designed to measure , which is defined as the temperature measured by an infrared radiometer operating typically at wavelengths 3.7–12 μm that represent the temperature within the conductive diffusion-dominated ocean sublayer at a depth of 10–20 μm (see Donlon et al. 2007). And one of the main design criteria of ISAR is to make the measurements autonomously when deployed on ships of opportunity (SOO).
The ISAR is a self-calibrating scanning radiometer with two internal calibration blackbodies (BBs), one BB at ambient temperature and the other BB heated to about 12 K above ambient. During one scan cycle, taking about 4 min to complete, the infrared detector, a Heitronics KT15.85D (KT15), views first the ambient BB, then the heated BB, next the sky, and finally the sea. Figure 2 shows the schematics of the scan mirror assembly (Fig. 2a) and the scan cycle (Fig. 2b), illustrating how self-calibration is performed for each scan cycle. Since no protective window is allowed to impede the sky view or sea view, the ISAR design incorporates a shutter that closes to prevent the ingress of rain or sea spray, thus protecting the optical components inside the casing but preventing temperature measurements during precipitation events. The shutter is controlled by a rain detector mounted close to the instrument, providing the control for autonomous operation of ISAR for up to 3 months.
To identify the sources of uncertainty in the estimates derived from an ISAR, the next few paragraphs outline the explanation by Donlon et al. (2008) of how the , as viewed by an ISAR, is calculated from the several instrument variables sampled during a scan cycle of the ISAR, to produce a single record. This is summarized in the temperature retrieval algorithm expressed in Eq. (10).
The ISAR internal calibration procedure must allow for the fact that, although the radiometer field of view is constrained by the field stop in front of the scan drum mirror, it is never possible to eliminate all stray radiation emitted by, or reflected from, inside the radiometer. The calibration of the externally viewed radiances is based on comparing the detector signal when viewing outward to that when viewing the calculated radiance from an internal BB cavity of known temperature, assumed to fill the same field of view as the external aperture. If it can be assumed that a large proportion p of the radiance reaching the detector is from the defined field of view, then the total radiance reaching the detector when viewing a target with radiance must be
where is the ambient stray radiation inside the sensor. It is further assumed that the KT15 signal (i.e., its digital output in counts C) is proportional to radiance over the range of brightness temperature (BT) encountered during ISAR operations. Thus,
where g is the internal gain of the detector in KT15 counts per radiance units. The subscript υ distinguishes between the four different viewing positions of the scan mirror during each cycle. Thus, , , , and represent the signal counts when viewing the sea, the sky, the ambient BB, and the heated BB, respectively. Note that the system is designed to sample the KT15 output a set number of times (typically between 10 and 50 but not necessarily the same for each view) before the scan mirror shifts to the next view. It is the average of all samples for a given view that is represented by . It is convenient to introduce the variables and , which are functions of , , , and recorded during the same ISAR measurement cycle and defined as
Similarly, the sky radiance can be expressed in terms of as follows:
Equations (6) and (7) allow the sea and sky radiances to be evaluated for each cycle from a knowledge of the radiances emitted by the two BB, without needing to know p, g, and . Moreover, as long as p, g, and remain constant within a measurement cycle of 4 min, this approach should allow any gradual drift in p, g, and to be accommodated without affecting the accuracy of the retrieved target radiance—that is, the sea view radiance, the sky radiance, or any other target presented to the radiometer, such as the laboratory calibration BB. It allows for some degradation of the scan mirror surface, which may reduce direct reflection and increase emission by the mirror surface itself so that the proportion of stray radiation increases. As long as the effect is identical for each view (sea, sky, and BBs during a particular measurement cycle), the retrieval of the radiance from external targets using Eq. (6) is not compromised. The radiances for the internal BBs in Eqs. (6) and (7) are calculated using
where is the radiance leaving the BB in the field of view of the radiometer, is the effective emissivity of the BB, and is the bandwidth-adjusted Planck function for the detector bandwidth, evaluated for the BB temperature () or the ambient temperature () internally within the ISAR, both of which are measured to a high accuracy by thermistors. Details of these thermistors are discussed in section 4a(1).
Each ISAR uses a set of polynomial coefficients determined by the KT15 spectral filter data to calculate the bandwidth-adjusted Planck and inverse Planck function. These coefficients were determined by T. Nightingale (2000, personal communication) and are stored in the instrument’s configuration file. The KT15 spectral filter has remained stable to date (2016), which has been verified by using an independent laboratory BB [Combined Action for the Study of the Ocean Thermal Skin second-generation blackbody (CASOTS II)] (Donlon et al. 2014) and the radiometer intercomparisons in 2009 (Theocharous et al. 2010).
Because the sea view radiance is a combination of emission from the sea surface and the reflection of sky radiance, we can write in general terms where ε is the emissivity of the sea surface,
Finally, by rearranging Eq. (9) and expressing radiances in the bandwidth limited form, the result is
where represents the band-averaged combined detector and wave band filter spectral response function and is the band-averaged . The terms and are the calibrated detector responses to the sea [Eq. (6)] and sky [Eq. (7)], respectively; and is the band-averaged emissivity at viewing angle θ. Since all the other terms and parameters in Eq. (10) are known or can be obtained from the instrument data record, the unknown value of can be evaluated by applying the inverse band-limited Planck function. This equation also provides the basis for identifying the measurement uncertainties discussed in the rest of the paper.
b. Operational deployments of ISARs based at Southampton
ISARs have been deployed on various vehicle ferries traversing routes between Portsmouth (United Kingdom) and ports in northern Spain, passing through the English Channel and the Bay of Biscay. ISARs were installed on the Peninsular and Oriental Steam Navigation Company (P&O) Pride of Bilbao (PoB) from March 2004 to September 2010, on the Brittany Ferries (BF) Cap Finistère (CpF) from October 2010 to July 2012 and on the BF Pont-Aven (PtA) since September 2012. Figure 3 shows the typical route of the PoB, which is broadly similar to the other ferries mentioned. To ensure nearly continuous operation of the ship-mounted installation, two ISARs are used to support the data collection, with one ISAR on the vessel at any time while the other instrument is being serviced and calibrated at Southampton University. The optical surfaces of an ISAR have an average lifetime of 3 months at sea before exposure to the marine atmosphere unacceptably degrades their measurement performance. The time series of data acquired from this installation is segmented into discrete deployments of up to about 3 months, during which an ISAR operates autonomously without any operator intervention.
The data used in this paper to demonstrate the newly developed uncertainty model are those acquired from deployment 32, which ran from 15 April 2011 to 20 July 2011 on the CpF. Figure 4 shows a time–latitude (Hovmöller) plot of the data collected. Gaps in the data are due to the instrument shutter being closed, and therefore not collecting data, because of bad weather.
c. Quality control of ISAR data
In previous publications of ISAR observations (e.g., Wimmer et al. 2012), the accuracy of the SST data retrieved from ISAR observations was expressed simply as a given estimated maximum measurement uncertainty, based on design considerations and empirical tests. Quality control consisted of laboratory validation runs in which ISAR viewed a gradually heating infrared radiation source calibrated with traceability to an International System of Units (SI units) reference standard (Donlon et al. 1999, 2014). Such tests were performed before and after each ship deployment while the ISAR remained sealed. Failure of the radiometer response to agree within the specified ±0.1-K accuracy between predeployment and postdeployment validation runs would lead to the rejection of all data from the whole deployment. This approach errs on the side of caution, and while it copes with the potential for degradation of the optical path through the ISAR, caused in particular by the effect of seawater spray or airborne particulates on the scan mirror, it also eliminates usable data from the ISAR record. Nonetheless, adopting a stringent approach ensured that the traceability of the radiometric calibration of the ISAR to an international infrared standard was maintained.
A serious weakness of this previous approach to data validation is that it verified the measurement quality only two times, at the start and end of each deployment of up to 3 months. Adjustments made for mirror degradation using the measured mirror performance (seeWimmer et al. 2012) assumed gradual changes during the deployment. This assumption implies an unknown extra uncertainty in each measurement. Furthermore, the assessment of data quality paid no attention to additional uncertainties introduced by environmental factors, such as sea state, whose impact could not be assessed in the laboratory validation runs.
3. An uncertainty approach for ISAR measurement quality
a. Rationale and basic concepts of an uncertainty approach applied to EO data
Global datasets from Earth observation satellites are used extensively to record the space–time distribution of environmental variables, including SST. They increasingly provide vital information for managing many human activities across the planet, they have become an essential input for weather and ocean forecasting, and they promise to be a source of reliable evidence concerning the extent to which the climate is changing. Given the variety of important tasks for which they are required, those using the data are entitled to know how much confidence can be placed in their accuracy. The international oversight of quality assurance (QA) for satellite Earth observation (EO) datasets has been adopted by CEOS, which is responsible for implementing the space-based observations planned for the Global Earth Observation System of Systems (GEOSS; CEOS 2015). To meet its QA obligation, CEOS has established the QA4EO.1 Its basic recommendation is simply stated, although its implementation is more challenging. The QA4EO principle is that data and derived products shall have associated with them a fully traceable indicator of their quality.
The purpose of the quality indicator (QI) is that it “shall provide sufficient information to allow all users to readily evaluate the ‘fitness for purpose’ of the data or derived product” (QA4EO Task Team 2010, p. 4). The intention is that users of an EO dataset should be able to interpret the QI in relation to the particular requirements of their applications. Users of EO data recognize that the accuracy of some data values will be better than others, and that the data provider already has some insight into the factors that result in this variable data quality. However, if the dataset consists of no more than the data values and a single overall validation statement about the whole dataset—for example, an error interval—then the data provider’s more subtle knowledge of variable reliability across the dataset is not communicated to the user. CEOS therefore recommended that the QI should be produced in the form of an uncertainty estimate that is calculated for, and attached to, each individual record. If the basis of the QI estimation is clear and understood by users, then it offers opportunities for users to be more effective in filtering data to meet particular quality standards required by their applications.
It is important to be clear about what is meant by the term uncertainty. The CEOS recommendations expressed in QA4EO are based on the generic approach developed by the metrology community (JCGM 2008). In this, the operational definition of “uncertainty of measurement” is expressed as a “parameter, associated with the result of a measurement, that characterizes the dispersion of the values that could reasonably be attributed to the measurand” (JCGM 2008, section 2.2.3). This definition avoids any reference to terms such as errors or true values, which, by the nature of measurement, are strictly unknowable. Nonetheless, the uncertainty can be used as an indication of the error in the estimated value of the measurand, or of the range of values within which the true value lies.
Ideally, the uncertainty should characterize the distribution of the dispersion of values that could be attributed to the measurand. However, if the distribution of uncertainty is assumed to be Gaussian and treated like a standard deviation, then the result is described as a “standard uncertainty.” It should be noted here that, in this paper, the estimated uncertainty for a variable implies that the estimate of the variable differs from its true value by less than the stated uncertainty in 95% of cases.
In the following sections, it is also helpful to distinguish between two general ways of evaluating uncertainties, according to the character of the uncertainty:
Type A: Uncertainties that must be estimated by using statistics, sometimes also referred to as random, since the uncertainties can be reduced by increasing the number of samples used in producing a single data record.
Type B: Uncertainties estimated from knowledge of component behavior or other information, sometimes also referred to as systematic, because they are not reduced by obtaining more samples.
A further way of characterizing the uncertainties is to split them into the following:
Measurement uncertainty: The uncertainty associated with the typical variability of the measured property, for example, for ISAR the variability of the brightness temperature of the sea view () and brightness temperature of the sky view ().
Instrument uncertainty: The uncertainty that the measuring instrument introduces regardless of the measured property.
b. Identifying the sources of uncertainty in measured by an ISAR
To estimate an uncertainty for each record, we first have to analyze the uncertainties of all the different parameters and sampled properties required to evaluate Eq. (10) for each record. This requires the terms in Eq. (10) to be unpacked from its derivation as summarized in section 2a. Figure 5 illustrates this schematically.
This flowchart in reverse is able to separate clearly the different sources of uncertainty from each other. For example, taking Eq. (10) as it stands, the top of the figure identifies four primary sources of error in the measurement of and , in the model of sea surface emissivity, and in the digital inversion of radiance to temperature (R2T). Each of these can be broken down further, especially the sea view and sky view radiances, to distinguish between uncertainties associated with various stages of the digital electronics, those related to the material properties of the ISAR (such as the emissivity of paint in the BB or the individual thermistor calibrations), and uncertainty in the radiance detector behavior. The colored boxes in Fig. 5 denote the fundamental sources of uncertainty, distinguishing between type A in blue and type B in red.
Typical estimates of the different sources of uncertainty are shown in Table 1. The justification for these estimates is presented in the following section. The right-hand column in Table 1 denotes the subsection in which the discussion can be found. Consideration of how these individual uncertainties interact to determine an overall uncertainty for each record from ISAR is presented in section 5.
4. Quantifying the sources of ISAR measurement uncertainty
This section examines the uncertainty contributions to the radiance calculations as shown in Fig. 5. Its specifically examines the internal calibration system and its main components, the thermistors, and the BB cavities.
To estimate the uncertainty of the internal calibration system, we first analyze the uncertainty of the thermistors used in the BB cavities, which ultimately define the baseline uncertainty for the whole instrument.
The thermistors used in the ISAR instrument are Yellow Springs Instrument Company (YSI) 46041.2 The YSI 46041 is a superstable (drift is less than 0.01 K per 100 months) interchangeable thermistor accurate to ±0.05 K (see Table 1, row 15). Furthermore, the thermistors in the internal BB are traceable to SI unit standards [National Institute of Standards and Technology (NIST), traceable calibration to ±0.05 K; Measurement Specialties 2008] and give the instrument its traceability. The YSI 46041 is part of a half bridge that has a Bourns 4808 (Bourns 2006) resistor network as the other part of the half bridge. The Bourns 4808 is a resistor with a thermal coefficient of ±100 ppm °C−1 (see Table 1, rows 9 and 10). The reference voltage to the thermistor half bridge is supplied by a Maxim MAX667 (Maxim 2008), which has a line regulation uncertainty of ±15 mV (see Table 1, row 7). The analog-to-digital converter (ADC) is an Adam 4017 (Advantech 1997): eight channels and 16 bits with a software-configurable range that was set to 5V. The accuracy is 0.1% of the range voltage (in the case of ISAR 5V) and has a zero drift of ±6 μV °C−1 (see Table 1, rows 5 and 6). Finally, all ADCs have a minimum of ±1 least significant bit (LSB) uncertainty (see Table 1, row 4).
2) Resistance to temperature approximation
A Steinhart–Hart approximation is used for the BB thermistors to estimate the measured temperature in the BB. The YSI 46041 data, as provided by Measurement Specialties (MS) (Measurement Specialties 2008), was used to calculate a polynomial fit through the data as shown in the top panel in Fig. 6. The middle panel in Fig. 6 shows the residuals for the Steinhart–Hart approximation in red and for the normal third-order polynomial fit in blue. The bottom panel in Fig. 6 shows a tenth-order fit for the estimation of the residuals, which has difficulty replicating the noise in the residuals around 20°C. The noise around 20°C is due to the coarse stepping of the original data supplied by Measurement Specialties (2008). The middle panel in Fig. 6 also shows that the Steinhart–Hart approximation has an increased uncertainty below 0° and above 75°C. While the upper limit is not very critical for this application, the lower end has to be considered if ice surface measurements are made. For the purpose of this uncertainty analysis, we used a fixed uncertainty for the Steinhart–Hart approximation of ±0.01 K (see Table 1, row 13). While using a fixed uncertainty is not ideal, the bottom panel in Fig. 6 clearly shows that a polynomial fit through the residuals would not estimate the uncertainties around 20°C correctly.
3) Internal blackbody emissivity
The internal ISAR BB cavities allow the instrument to calibrate the KT15 signal every scan cycle. Figure 7 shows the geometry of the BB and the location of the three thermistors with which each blackbody is fitted. The BB use a reentrant cone and a partially closed aperture design which, combined with a high-emissivity surface finish (Nextel velvet black) and critical internal geometry, ensure that the blackbody cavities have an emissivity of >0.999 in the thermal infrared wave band (Berry 1981). Donlon et al. (2008) estimated the internal BB emissivity to be 0.9993. We revisited the calculation and estimated an uncertainty for the emissivity by taking aging of the paint and the uncertainty of the Berry (1981) estimation into account. The result of this calculation gives an emissivity of 0.9993 ± 0.000 178 (see Table 1, row 11). This is a smaller uncertainty than the analysis for the Marine Atmospheric Emitted Radiance Interferometer (M-AERI), which estimated the BB emissivity uncertainty as ±0.0008 (Best et al. 2003); however, ISAR uses a reentrant cone design and significantly closed aperture that is less susceptible to the aging BB paint.
b. KT15, T2R, and R2T conversion
The detector used in the ISAR instrument is a Heitronics KT15.85D with a temperature range from −100.0° to 50°C. The KT15 is linear in radiance but not in temperature. Figure 8 shows the detector response in relation to the target temperature. The detector accuracy is ±0.5 K + 0.7% of the temperature difference between the detector case and the target temperature. The long-term stability of the detector is better than 0.01% K month−1 (Heitronics 2000) (see Table 1, rows 1 and 3). The spectral response of a number of KT15 used for the existing ISAR instruments is shown in Fig. 9. Because of the differing spectral responses between the different KT15s used for each individual ISAR, it is necessary to estimate the R2T and temperature to radiance (T2R) conversion polynomials separately for each KT15. The R2T and T2R conversion polynomials were estimated by T. Nightingale (2000, personal communication), and their uncertainty is quoted as ±1 mK (see Table 1, row 14).
c. Seawater emissivity
The emissivity of the sea surface not only has type B uncertainties but also a type A component, determined by the changing environmental conditions, such as the ship’s roll or sea state. However, because the emissivity value itself is not determined by an ISAR measurement, it seems more appropriate to discuss the seawater emissivity as a type B uncertainty.
Various studies have looked into the variability of seawater emissivity and its dependence on view angle and sea state. Wu and Smith (1997) and Masuda et al. (1988) present a theoretical approach for estimating the seawater emissivity, with Niclòs et al. (2009) giving a simplified equation for the atmospheric windows in the infrared region. The emissivity model used by the (A)ATSR Reanalysis for Climate (ARC) project (Embury et al. 2012) is based on Watts et al. (1996), Masuda et al. (1988), and Wu and Smith (1997), with some improvements from Newman et al. (2005). Newman et al. (2005) revisits the salinity and temperature dependence of sea surface emissivity (ε), and while the salinity dependence is captured very well in Watts et al. (1996), Masuda et al. (1988), and Wu and Smith (1997), the temperature dependence is less well characterized. However, this is mainly an issue in the 750–850 cm−1 wavenumber region, which is below the ISAR detectors filter region (870–1050 cm–1) and at high viewing angles (>50°).
Hanafin and Minnett (2005) and Niclòs et al. (2005) estimated the surface emissivity from in situ measurements, and Hanafin and Minnett (2005) found that at a view angle of 55° an error of up to 0.7 K can be introduced into the shipboard radiometer SST calculation. Both Hanafin and Minnett (2005) and Niclòs et al. (2009) show that ε does not decrease as much with increasing wind speed as Watts et al. (1996), Masuda et al. (1988), and Wu and Smith (1997) predict. However, Niclòs et al. (2005) shows that the wind speed dependence of emissivity is near zero at wind speeds up to 10 m s−1. Masuda (2006) revisited the Masuda et al. (1988) and Wu and Smith (1997) calculations and added a surface-emitted surface-reflected radiation (SESR) term that results in similar ε as Hanafin and Minnett (2005). The SESR term has very little effect below a 40° view angle and increases from a view angle of 50°, showing a maximum value of 0.03 at an view angle of 80° to be added to ε.
To investigate the impact of the seawater emissivity on the overall uncertainty budget of the ISAR SST estimation, a simple model of the ISAR SST estimation was used. The model in Eq. (11) is based on Eq. (10) and simply varies the input parameters in a range as outlined in Table 2. The value ranges are typical ranges for the , , and ε. The step value in Table 2 was determined by the integration time of the model and has no effect on the result,
Figures 10 and 11 show the output of the simple emissivity model. The standard deviation (std) is calculated over the emissivity range as stated in Table 2 for each and . Both figures show that the uncertainty converges to 0.0 the closer and are to each other. This is to be expected from Eq. (11) and fits well with the theory. The plots in Figs. 10 and 11 also show that the uncertainty introduced by the modeled ε is in a similar range to that measured by Hanafin and Minnett (2005).
Because of the view angle dependence of ε, the actual view angle has to be considered for the uncertainty budget. This was achieved by using the ship’s roll angle as measured by the ISAR instrument’s own pitch and roll sensor to calculate the actual view angle of the ISAR instrument. Donlon and Nightingale (2000) has also shown that because of the ship movement, not only the changing emissivity but also the resulting mispointing of the sky view has to be considered in the uncertainty budget. Donlon and Nightingale (2000) give a deviation of 0.025 K for a sky view mispointed by 10°. We used the Niclòs et al. (2009) model to calculate the change in ε with changing view angles and calculated the Niclòs et al. (2009) model over varying wind speeds from 0 to 20 ms−1. The values for Niclòs et al. (2009) are calculated from Advanced Spaceborne Thermal Emission Reflection Radiometer (ASTER) (Baldridge et al. 2009) seawater emissivity values integrated over the ISAR spectral window. The for the ISAR view angle of 25° is calculated as 0.9916.
By taking all of the above-mentioned considerations into account, the seawater emissivity could range from 0.92 to 0.9916. By looking at the ISAR data, we get an uncertainty for the emissivity ranging from ±10−4 to ±10−7 depending on the conditions (see Table 1, row 12).
5. Modeling uncertainty propagation through the ISAR processing chain
Having considered all the individual sources of uncertainty inherent in the measurement of using an ISAR, we now describe the method used, for combining these uncertainties to a single uncertainty value.
The ISAR instrument is designed as a self-calibrating radiometer, where some of the uncertainties are reduced by the internal calibration process and therefore adding them by summing the squares of the individual uncertainties identified in Table 1 is not appropriate. There seems to be no easy alternative to propagating the uncertainties identified in Table 1 through the ISAR SST processor, where they interact, to obtain the total uncertainty associated with each measurement. This section first introduces a software package adopted to carry out this procedure. It then explains how the different types of uncertainty are entered into the package. Finally, it critically assesses, as a test case, the uncertainty estimations generated for a particular week of data from the ISAR archive.
a. The software package used for evaluating uncertainty estimates
The ISAR postprocessing software was updated with a preexisting Python package (Lebigot 2012) to allow for the processing of the uncertainty and the propagation of uncertainty values through the processing chain. Lebigot (2012) states that in this package the value of a variable with uncertainty attached is treated as a probability distribution. This is not restricted to a normal (Gaussian) distribution and can be any kind of distribution. These probability distributions are reduced to two numbers: a nominal value and a standard deviation. Furthermore, the uncertainty package calculates the standard deviation of mathematical expressions through the linear approximation of error propagation theory. This is why this package also calculates partial derivatives. The standard deviations and nominal values calculated by the package are thus meaningful approximations as long as the functions involved have precise linear expansions in the region where the probability density of their variables is the largest. It is therefore important that uncertainties be small. Mathematically, this means that the linear terms of functions around the nominal values of their variables should be much larger than the remaining higher-order terms over the region of significant probability.
The Python uncertainty package was applied using the mode and the variance of each parameter relevant to the ISAR calculation. This was done in order to reach the most statistically robust solution for the linear uncertainty model. The next section describes the main steps that the updated ISAR postprocessing software performs to derive an uncertainty for each estimate. Also, the ISAR uncertainty is an estimate of the variable that differs from its true value by less than the stated uncertainty in 95% of cases, or in other words the coverage factor . The coverage factor is used to scale the standard uncertainty, which can be thought of as equivalent to one standard deviation, to the different confidence interval than the equivalent of 66.7%. The scaled uncertainty is called the expanded uncertainty and is derived by multiplying the standard uncertainty with the coverage factor (Bell 2001).
b. Steps in the uncertainty calculation
Making use of all the individual uncertainties discussed in section 4, the overall uncertainty is estimated by propagating the individual uncertainties through the ISAR SST processor, adding the uncertainties to each step:
The thermistors’ uncertainties are calculated by first assigning the ADC uncertainty to the measured voltage. Then the resistance of the thermistor is calculated by taking the uncertainty of the reference voltage and the reference resistor into account. The resistance is converted into a temperature by using the Steinhart–Hart approximation, taking the uncertainty of the approximation into account. The uncertainty resulting from this temperature calculation is then combined with the manufacturer-quoted thermistor uncertainty of ±0.05 K and the variability of the thermistor temperature during a BB target view to derive the thermistor uncertainty used in the temperature-to-radiance transformation.
For the detector signal, the ADC uncertainty and the KT15 temperature dependence, using the temperature of the KT15 case thermistor (which is calculated using the approach described above) and the variability over each target view, are used to calculate the KT15 signal uncertainty.
Then the internal BB radiances with uncertainties are calculated by using the BB and window thermistor temperatures together with the internal BB emissivity uncertainty by using Eq. (8).
Now the sky view and sea view radiances can be calculated by first calibrating the detector signal together with its uncertainties as described in Eqs. (3) and (4), respectively, and then using the calibrated detector signal to calculate sky and sea radiances with the associated uncertainty from the BB radiances as shown in Eq. (6). The BB radiances for Eq. (6) are estimated as described in step (i).
The term ε is estimated by using the Niclòs et al. (2009) model averaged over a wind speed range of 0–20 m s−1 with the view angle calculated from the maximum ship roll during the sea view and sky view.
The penultimate step is calculating the SST with a total uncertainty by using the sea and sky radiances and their associated uncertainties together with ε with its uncertainty in Eq. (10).
The final step is to add the CASOTS II BB uncertainty (Donlon et al. 2014) in quadrature to the uncertainty as calculated by the previous steps. This step is necessary only because of the different responses of the half-bridge resistor circuit of individual ISARs. Without this step the traceability is achieved through the BB thermistors.
The type A uncertainties are estimated in the averaging calculation at each of the BB views and the sea view and sky view. The averaging at each of these views is necessary to reduce the KT15 noise.
c. Examples of uncertainty estimates in operational ISAR datasets
Figure 12 shows the total uncertainty as calculated for ISAR data collected between 15 and 22 July 2011 on the CpF. Also shown in Fig. 12 are the uncertainty split between type A and type B (bottom-left panel) and the split between the instrument and measurement uncertainty (bottom-right panel).
To estimate the type A–type B split and the measurement and instrument uncertainty, the ISAR SST processor was run four times: one time each with one of the type A, type B, or instrument uncertainty switched off and once with all parameters switched on for the total uncertainty. For the separation of type A and type B uncertainties, the definition as given in section 3a is followed. To separate the measurement and instrument uncertainty, the target detector views’ (sea view and sky view) uncertainty were set to 0.0 to obtain the instrument uncertainty. The measurement uncertainty was calculated as the difference between the instrument uncertainty and the total uncertainty. Calculating every realization of the uncertainties from first principles is computationally expensive and for future versions of the ISAR SST lookup tables might be used for some of the calculations to speed up the uncertainty estimation.
The target uncertainty for ISAR was to be below 0.1 K as quoted in the instrument specification (Donlon et al. 2008). While Fig. 12 shows that the total uncertainty is in most cases below 0.1 K, it is not always the case. From the bottom-right panel in Fig. 12, it can be seen that the high uncertainty values coincide with high values of measurement uncertainty, showing that the high uncertainty values can be attributed to the ship’s roll and variability in the sea view and sky view of the KT15. To investigate the contribution of sources of uncertainty further, Fig. 13 shows the uncertainty data plotted against a few key parameters and a histogram of the uncertainty. The histogram shows that over 90% of the uncertainty estimates are below 0.1 K. The coloring of data points in blue and red in Fig. 13 is to distinguish between those cases where the uncertainty is associated with a data record for which lies between the ambient and hot BB (red dots) and those where was lower than the ambient BB (blue dots). The coloring was applied in order to investigate whether there is a dependence of the uncertainty on what the measured sea temperature was in relation to the internal calibration target temperatures. While Fig. 13b shows a small dependency of estimated BT uncertainty in relation to the ambient BB; it has no effect whether or not the measurement is between the two BBs. The main driver for the small dependency shown in Fig. 13b is the emissivity of the internal BB. Figure 13f shows that there is also a dependency of the uncertainty on the , especially at very low , which is expected as this enhances the effect of the ε uncertainty. Figure 13c shows that much of the higher uncertainty values occur in port as well as in the Bay of Biscay.
Figure 14 shows a geographical map of the same data as Fig. 13c, although some of the high uncertainties in port are not so visible in Fig. 14 because the plotting of the dots on top of each other masks some of the higher uncertainties. The ISAR uncertainty does not seem to be dependent on the or the , as confirmed by Figs. 13e and 13h, respectively.
The uncertainty estimates shown in Figs. 12–14 is the first attempt to do this, and the choice of dependencies to explore has been influenced by previous work on infrared radiation uncertainties presented in the literature. For example, the variation of the seawater emissivity (section 4c) was mainly estimated by the ship’s roll, as measurements of wind and sea state were not available for the whole of the ISAR deployments. Another aspect deserving further work is the instrument mispointing uncertainty, as shown by Donlon and Nightingale (2000). However, because the ISAR data used for the uncertainty estimation are collected on a semioperational basis for the AATSR validation contract3 (Wimmer et al. 2012), changes to instrument configuration such as scanning over a number of view angles could not easily be accommodated.
The reliance on previous work is also evident in the type A–type B split as shown in Fig. 12. While the separation into type A and type B uncertainties seems to give a reasonable estimate of the random and systematic uncertainty component, there are situations where some of the type A uncertainty appears as type B uncertainty. This is partly due to the fact that some type B uncertainties have both a type A and a type B component, but because the references used for the uncertainty budget did not separate the two components, they have to be classified as type B uncertainties. The differentiation of the total uncertainty into measurement and instrument uncertainty seem to show fewer cross-correlation issues than the type A and type B uncertainties, although they are still apparent in some situations.
6. Discussion on the utility of the ISAR uncertainty model
The updated ISAR processor was applied to the whole ISAR data archive (2004–15) and the results are shown in the Hovmöller plot in Fig. 15, a plot of all the ship tracks in Fig. 16 and the histogram of the uncertainty distribution is shown in Fig. 17. The gaps in the Hovmöller plot are periods where either the weather was too rough to measure data and the ISAR shutter was closed or the ferry was in refit. The ISAR data archive, which is available at National Environment Research Council (NERC) Designated Data Centre for Earth Observation (NEODC), includes all the uncertainties: total, type A, type B, instrument, and measurement uncertainty. Also included in the data are the values of ε, roll and pitch of the vessel, , and to allow the user to evaluate each measurement and to determine its usefulness for the user’s application.
The results of applying the uncertainty analysis to the ISAR data archive (Fig. 17) show that for 77.6% of the individual ISAR records, the uncertainties are below the target uncertainty of 0.1 K. Another 17.2% of the ISAR records have an uncertainty between 0.1 and 0.2 K, which can be acceptable data for certain applications but might not be sufficiently accurate for validation of CDR satellite datasets. The remaining 5.2% of data have a fairly high uncertainty, which at times can be explained by environmental factors such as passing the Ushant tidal front or instrument exceptions, although further work to understand these cases is needed. Also, the higher uncertainties seem to be more frequent in the last three years of the deployments, which is coincidental with the change of installation on the PtA, but this needs further investigation to establish whether it is merely coincidental or an effect of the installation on the PtA. While certain regions on the ship track are prone to higher variability, such as the Ushant region, the records do not show consistently higher uncertainties as Figs. 15 and 16 demonstrate.
The new ISAR uncertainty estimates presented in this paper satisfy the QA4EO recommendations, since their method of production adheres to sound and metrological principles (JCGM 2008). The ISAR uncertainty model provides a quality indicator for each measurement as required by QA4EO if ISAR data are to be used for the validation of satellite-derived SST CDRs (Minnett and Corlett 2012).
However, while the ISAR uncertainty provides a quality indicator, in its current form there are some weaknesses that have to be discussed. Because the ISAR uncertainty model used the ISAR instrument equation to propagate the uncertainness of the different parts to the overall uncertainty, it is not a simple task to identify the impact each individual uncertainty component has on the final uncertainty value. Another issue of the method of propagating uncertainties is that only the cross correlations and therefore the covariances of components in the subequations are addressed and some of the overarching cross correlations are missing from the ISAR uncertainty model; having said that, there are a few areas in the model that can be identified as major contributors or limitations of the model.
First, the sea surface emissivity and its accurate knowledge have a large impact on the uncertainty estimation; however, because the sea surface emissivity is calculated from models (Masuda 2006; Niclòs et al. 2009) and the wind and sea state have to be estimated and are not measured, the impacted of the sea surface emissivity might not be an accurate representation. Second, the model is highly reliant on the BB thermistors and in general their uncertainty is well defined; however, any bias introduced by the BB thermistor will not be detected by the uncertainty model and has to be corrected by using the pre- and postdeployment calibrations with the CASOTS II (Donlon et al. 2014). Another factor that has to be corrected with the use of the CASOTS II is the mirror degradation (see Wimmer et al. 2012), as the ISAR uncertainty model includes only the increase in the detector signal variability, not the change this causes to the internal calibration gain. There are some residual effects of the corrections that are currently not included in the ISAR uncertainty model, as it is not a simple matter to quantify those residual effects.
One of the reasons why it is difficult to quantify the effects discussed above is that the absolute value of the uncertainty has not yet been independently verified. To accomplish this requires a comparison of two or more infrared radiometers at sea. Previous radiometer comparisons at sea (Barton et al. 2004; Donlon et al. 2008) show a standard deviation ranging from 0.09 to 0.15 K between the values with biases ranging from 0 to 0.04 K. This would suggest that the ISAR uncertainty values are slightly lower than expected from these intercomparisons and that the histogram in Fig. 17 should be centered closer to the 0.12-K value than it is now, if we assume a Gaussian distribution. The reason for the potential underrepresentation might be difficult to establish, but it is most likely the sea surface emissivity uncertainty. However, it has to be mentioned that both of these intercomparisons were short in time: Barton et al. (2004) lasting 2 days, resulting in 144 data points; and Donlon et al. (2008) lasting 23 days with no mention of the amount of data points. While these comparisons are the best datasets we currently have, they are a very small sample size and do not cover all potential environmental conditions encountered by radiometers at sea, which does limit the value of these datasets. Furthermore, the comparisons looked at matchup pairs between two instruments, even though five (Barton et al. 2004) and three (Donlon et al. 2008) radiometers participated, and not a three-way uncertainty analysis (e.g., O’Carroll et al. 2008; Tokmakian and Challenor 1999), which would allow for attributing individual uncertainties to instruments. While an intercomparison lasting a few months with three or more instruments would be very useful to analyze the absolute uncertainty value, it is a nontrivial and expensive enterprise to organize such a campaign. Also without using ancillary data information, such as wind or sea state, which is not included in the ISAR data record, it might be difficult to improve on the current uncertainty values. However, the National Physical Laboratory (NPL)-led ESA project Fiducial Reference Measurements for Validation of Surface Temperature from Satellites (FRM4STS) is working to address side-by-side intercomparisons and hopefully such data will be available in the future.
ISAR measurements have been used in satellite SST validation since 2004. These measurements are traceable to SI units and show high accuracy (Wimmer et al. 2012), but in order to adhere to metrological standards (JCGM 2008) and to be able to demonstrate that satellite SST can be regarded as an ECV, a quality indicator of the data was needed. The ISAR uncertainty model estimates a quality indicator for each measurement and thus gives confidence that ISAR data are suitable as Fiducial Reference Measurements (FRM). The ISAR uncertainty model demonstrates that ISAR measurements in the Bay of Biscay and the English Channel are within ±0.1 K for 77.6% of all ISAR data records, with only 5.2% of the data having a higher uncertainty than 0.2 K. The absolute value of the uncertainty seems lower than previous work on a shipborne radiometer comparison (Barton et al. 2004; Donlon et al. 2008) suggests; however, further comparisons with radiometers where each has an uncertainty model are needed to establish and rectify potential shortcomings of the ISAR uncertainty model. We discussed some of the shortcomings of the uncertainty model and potential future work to improve these areas; nonetheless, the uncertainty model provides a clear indicator to assess the quality of individual records. Currently, the ISAR is the only shipborne radiometer that provides an uncertainty model, and until other shipborne radiometers provide a similar model, the verification of the absolute uncertainty values is a nontrivial matter.
The ISAR uncertainty model should not only give confidence in the quality and accuracy of measurements but also demonstrate that satellite SST measurements can achieve CDR quality and be fully traceable to SI unit primary standards.
This work was funded by the U.K. Defra and DECC through Contract 2004-03-002/CPEG 10, managed by Space ConneXions Ltd. We acknowledge the generosity of P&O and BF in allowing the ISAR instrument to be deployed on the PoB, CpF, and PtA, and thank the masters and crew of all the vessels since 2004 for their ongoing assistance. We are grateful to Hugh Kelliher of Space ConneXions Ltd., Gary Corlett and John Remedios of the University of Leicester, and Tim Nightingale of Rutherford Appleton Laboratory, partners in the AATSR Data Exploitation Contract, for the ongoing constructive criticism of this work. We also acknowledge the outstanding technical support provided by Raymond Holmes, Gary Fisher, and colleagues in the workshop of the School of Ocean and Earth Science, University of Southampton, in the National Oceanography Centre, Southampton.
Advanced Along-Track Scanning Radiometer
Analog digital converter
(A)ATSR Reanalysis for Climate
Advanced Spaceborne Thermal Emission Reflection Radiometer
Brightness temperature of the sky view
Brightness temperature of the sea view
- CASOTS II
Combined Action for the Study of the Ocean Thermal Skin second-generation blackbody
Climate data record
Committee on Earth Observation Satellites
Department of Energy and Climate Change
Department for Environment, Food and Rural Affairs
Essential climate variable
European Space Agency
Fiducial Reference Measurements
Fiducial Reference Measurements for Validation of Surface Temperature from Satellites
Global Earth Observation System of Systems
Global positioning system
Infrared Sea Surface Temperature Autonomous Radiometer
Least significant bit
Marine Atmospheric Emitted Radiance Interferometer
NERC Designated Data Centre for Earth Observation
National Environment Research Council
National Institute of Standards and Technology, United States
National Physical Laboratory, Teddington, United Kingdom
Pride of Bilbao
Peninsular and Oriental Steam Navigation Company
Quality Assurance Framework for Earth Observation
Radiance to temperature
Surface-emitted surface-reflected radiation
- SI units
International System of Units
Ships of opportunity
Sea surface temperature
Sea surface temperature at the skin interface
Temperature to radiance
- Type A
Type A uncertainties
- Type B
Type B uncertainties
Yellow Springs Instrument Company
The YSI temperature division was purchased by Measurement Specialties from YSI Incorporated (Ohio) in April 2006. However, for the purpose of this paper we still call the thermistor a YSI 46041.
The ISAR project has been funded since 2004 by the Department for Environment, Food and Rural Affairs (Defra) [now the Department of Energy and Climate Change (DECC)] to provide in situ data for the AATSR validation.