1. Introduction

An accurate representation of precipitation is necessary for an integrated understanding of Earth’s climate system. Yet large uncertainties in precipitation estimates exist over the Southern Ocean (SO) (Boisvert et al. 2020; Manton et al. 2020), which is a challenging region because of sparse surface sites, large sea ice masses, and strong winds and waves (Burdanowitz et al. 2019; Boisvert et al. 2020; Siems et al. 2022). This has hindered our ability to understand a wide range of climate and atmospheric processes in this region, as well as their far-reaching impacts through teleconnections (Bodas-Salcedo et al. 2014; Ceppi et al. 2016; Gettelman et al. 2019).

Given the scarcity of in situ data, satellites have been the foremost tools for detecting and estimating precipitation over remote regions. In fact, a global estimation of precipitation is only possible via satellites, due to their ability to provide near-global and near-continuous coverage (Bumke et al. 2016). However, satellite-based products are still only indirect measures of precipitation using passive and/or active remote sensing instruments (Huffman et al. 2019a). The Integrated Multi-satellitE Retrievals for Global Precipitation Measurement (IMERG) product is currently one of the most advanced satellite precipitation products available, calibrating passive retrievals from across the GPM constellation against active Ku-band radar retrievals from the GPM Core Observatory satellite (Skofronick-Jackson et al. 2017). Products such as these have predominately been developed and calibrated using observational data over the Northern Hemisphere (NH), making them prone to large uncertainties over the pristine SO (Behrangi and Song 2020), where the dynamical and microphysical features of the precipitation systems differ (Huang et al. 2015).

Reanalysis products, such as the fifth major global reanalysis produced by European Centre for Medium-Range Weather Forecasts (ECMWF) (ERA5; Hersbach et al. 2020), are an attractive alternative source of precipitation estimates. Unlike satellite-based products, reanalysis precipitation is derived from model simulations, whose accuracy is constrained by model dynamics, model physics, and assimilated information (Hersbach et al. 2020). Small-scale and local processes, such as convection, are often missed or poorly represented due to the limited model resolutions and various assumptions used in the parameterization schemes (Behrangi et al. 2016).

The large discrepancies between the reanalysis and satellite-based precipitation estimates over the SO have translated into inconsistencies in precipitation trends and thermodynamic phase classifications (Behrangi et al. 2014, 2016; Boisvert et al. 2020; Manton et al. 2020). Manton et al. (2020) showed that trends of precipitation monthly anomalies derived from several widely used products vary, even with opposite signs over different sectors of the SO. Snowfall precipitation intensity estimates between reanalysis and satellite products also diverge, especially at mid- and high latitudes of the SO (Boisvert et al. 2020). These challenges necessitate a comprehensive assessment of the reanalysis and satellite products across the range of SO precipitation mechanisms (Khan and Maggioni 2019).

Precipitation evaluation studies using surface measurements over the SO are often limited to a few island sites, with the records collected from Macquarie Island (MAC) being of exceptional value given both the quality and extent, dating back to 1948. An evaluation of the ERA-Interim product using MAC rain gauge data showed that ERA-Interim underestimated the annual precipitation from MAC by about 6.8% (Wang et al. 2015). It was also found that ERA-Interim overestimated the precipitation in the midlatitude cyclones and fronts and underrepresented the precipitation away from these systems, errors that were largely fixed by ERA5 (Lang et al. 2018; Wang et al. 2015; Boisvert et al. 2020). More recently, Tansey et al. (2022) used a blended set of in situ and remote precipitation estimates to evaluate the CloudSat precipitation products, finding that in the MAC area, CloudSat estimates tend to underestimate light liquid precipitation frequency and intensity and mixed-phase precipitation frequency. Despite these efforts, a more systematic assessment of the precipitation products, under the varying meteorology across the broader extent of the SO, is largely absent.

While ship-based data from research vessels (R/Vs) and ocean buoys represent another useful set of measurements, the rain gauges of standard design being conventionally employed can often be biased by sea spray, intense winds, and spatial and temporal coverage (Klepp et al. 2010, 2020). The Ocean Rainfall And Ice-Phase Precipitation Measurement Network (OceanRAIN) disdrometer, which was specifically designed to reduce uncertainties arising from rough oceanic and atmospheric conditions, offers much greater confidence in ship-based observations (Klepp et al. 2010). Initiated in 2009, the OceanRAIN project is arguably the only systematic long-term shipboard precipitation data collection effort to date (Klepp et al. 2020; Klepp 2020). The observations have been used to evaluate a variety of satellite and reanalysis surface precipitation products, with a heavy focus on the NH oceans and bulk precipitation statistics (Bumke et al. 2012, 2016; Burdanowitz et al. 2019). Since the first deployment over the SO in 2016, OceanRAIN has provided the scientific community with unique and high-quality precipitation information in this region. Using approximately 790 h of OceanRAIN disdrometer observations, Montoya Duque et al. (2022) found that precipitation over the SO is dominated by very light and light precipitation (below 1 mm h⁻¹) and that the high-latitude region commonly has mixed-phase and snow precipitation.

Another significant advance in recent years has been the availability of a dual-polarization C-band weather radar (OceanPOL) over the SO. Onboard the R/V Investigator (operated by the Australian Marine National Facility), the OceanPOL commenced its mission in 2018, making extensive and near-continuous measurements across the Australian sector of the SO through multiple cruises. Since its deployment, the OceanPOL has produced a wealth of fresh precipitation details with unprecedented spatiotemporal coverage, making the dataset useful for evaluation purposes.

Motivated by these emerging opportunities, this research aims to perform a systematic evaluation of two widely used precipitation products over the SO, IMERG and ERA5, using the OceanRAIN disdrometer and OceanPOL radar observations collected from multiple recent field campaigns. We also extend the analysis to examine precipitation estimates under various synoptic conditions, using the recently established clustering synoptic classification by Truong et al. (2020) and Montoya Duque et al. (2022). The research is guided by two questions: How accurate are precipitation estimates over the Australian sector of the Southern Ocean compared to ship-based measurements? How does the accuracy vary under the various synoptic conditions? The data and methods are described in section 2, followed by the results in sections 3–6, and discussion and concluding remarks in section 7.

2. Data and methods

a. Data

1) Surface-based and remote sensing measurements

The R/V Investigator (CSIRO 2022) has deployed an OceanRAIN disdrometer (ODM470) (Klepp et al. 2018; Protat and CSIRO 2020) since 2016 and a C-band dual-polarization weather radar (OceanPOL) since 2018 (Louf et al. 2019) over the SO and Australian coast (Table 1). A total of six research voyages (181 days) were undertaken over the Australian sector of the SO and the Antarctic coast between the austral summer season or early fall 2016 and 2018 (Fig. 1). Two of the voyages were part of the Clouds, Aerosols, Precipitation, Radiation, and Atmospheric Composition over the Southern Ocean (CAPRICORN) field campaigns (McFarquhar et al. 2021), where the observations were enhanced by a suite of cloud–aerosol–radiation measurements such as a W-band cloud radar (Table 1), providing further insights into the case analysis presented in section 3.

Table 1.

Remote sensing and in situ instruments deployed on the R/V Investigator that are employed in this study.

View Table

View Full Size

Trajectories of research voyages undertaken over the SO by the R/V Investigator between January 2016 and March 2018: Blue, 7 Jan–7 Feb 2016; orange, 14 Mar–16 Apr 2016 (CAPRICORN I); green, 14 Jan–5 Mar 2017; red, 17–27 Mar 2017; purple, 11 Jan–22 Feb 2018 (CAPRICORN II); brown, 3–21 Mar 2018.

Citation: Journal of Applied Meteorology and Climatology 62, 11; 10.1175/JAMC-D-23-0039.1

• Download Figure
• Download figure as PowerPoint slide

Compared to other existing disdrometers that are not designed for shipboard operation, the ODM470 was specially developed to meet all-weather shipboard requirements and has been shown to have improved performance under strong wind conditions (Klepp et al. 2018). In this study, we use the OceanRAIN v2.0, which provides additional ship records and radar-related variables that were not available in previous versions (Klepp 2020). However, quantitative measurements of snow and mixed-phase precipitation are still challenging with the ODM470 and involve large uncertainty, especially during strong precipitation events. The liquid-to-solid ratio from mixed-phase events remains poorly quantified, and snow shape and density assumptions do not necessarily represent the snow variability (Klepp et al. 2010, 2018). After removing obvious outliers from the ODM470 precipitation rates, there remains a small number of samples with suspiciously large intensities (above 50 mm h⁻¹), which are related to one precipitation event that occurred at high latitudes during mixed-phase periods. These data should be treated with greater caution. The hourly thermodynamic phase is estimated based on the hourly frequency of each phase. The liquid or snow phase is defined when the respective frequency of occurrence is at least 90%; mixed phase is assigned when liquid or snow is between 10% and 90%.

OceanPOL weather radar product v2020, provided by the Australian Bureau of Meteorology (BoM), includes estimates of precipitation rate produced from the dual-polarization rainfall algorithm (see Thompson et al. 2018). The algorithm employs information from the horizontal radar reflectivity and differential reflectivity. The cross-correlation coefficient ρ[HV(0)] is used for quality control since it indicates the presence of non-Rayleigh scatters, the similarity between scatters’ phase, and orientation. Compared to single-polarization radars, dual-polarization information allows for better detection of the melting layer, quantitative rainfall estimates, and precipitation type classification (Thompson et al. 2015). The Thompson et al. (2018) algorithm was calibrated for liquid precipitation using data from two tropical islands, which may not be optimal for the SO environment. Additionally, a sensitivity analysis of the OceanPOL radar histograms suggests that at a long range (greater than about 100 km), reflectivities below 10 dBZ are not measured, thus limiting the skill of rainfall detection and estimates for drizzle/light precipitation events. To avoid sea-clutter contamination, we use the gridded precipitation rate estimates produced at an altitude of 500 m, the lowest altitude representing near-surface precipitation. Further, we exclude all values within a 5-km radius from the radar location before interpolating the data onto a 0.25° × 0.25° grid using the nearest gridpoint method. In the operational dataset, rainfall (i.e., liquid precipitation) estimates were produced for all valid radar grid points regardless of the precipitation phases. Thus, we identify rain grids based on the microphysical classifier (Thompson et al. 2014), a ρ[HV(0)] value above 0.9, and when the corresponding hourly ODM470 phase classification indicates either liquid or no precipitation at the ship location.

2) Satellite data

The satellite-based estimates of precipitation from the IMERG product are examined in this study. The IMERG dataset is currently the most advanced remote sensing precipitation product with a global coverage (Huffman et al. 2019a). The IMERG algorithm intercalibrates, merges, and interpolates microwave, microwave-calibrated infrared, and precipitation gauge data and other data from suitable spaceborne sensors to produce a gridded dataset with a spatiotemporal resolution of 0.1° and 30 min (Huffman et al. 2019a, 2019c). In the IMERG product, precipitation estimates can come from passive microwave (PMW)-only overpasses, infrared-only (IR) overpasses, interpolated values using a morphing-only scheme based on MERRA-2, or an interpolation between PMW-IR or PMW-morphing (herein defined as mixture; see online supplemental material) (Li et al. 2022). The IMERG precipitation algorithm is an upgrade from the Tropical Rainfall Measuring Mission (TRMM) product, making IMERG largely tuned for tropical rainfall (Huffman et al. 2019a; Protat et al. 2019b). Unlike TRMM, however, IMERG provides information at higher latitudes (full coverage between ±60° and partial coverage poleward to this latitude), precipitation phase classification, and improved sensitivity to light rain. Field campaigns such as CAPRICORN II had aimed at improving GPM precipitation products, among others. When available, the OceanRAIN product provides freshwater flux data employed by IMERG to improve the ocean precipitation estimates (Klepp et al. 2020; McFarquhar et al. 2021). In this study, we use the rain gauge–calibrated precipitation rate estimates (precipCal) and the probability of liquid precipitation (%) for phase classification by using similar thresholds as for the disdrometer data [see section 2a(1)]. The final run of the level 3 product (level 3 V06B) includes all satellite overpasses, providing a complete picture of precipitation around the globe (Huffman et al. 2019a). This is especially useful for areas where not a lot of ground-based data are available, such as the SO.

The Himawari-8 satellites, operated by the Japan Meteorological Agency (JMA), carries the Advanced Himawari Imager (AHI) instrument, which is a 16-channel multispectral imager capturing visible and infrared images. In section 3, we supplement the precipitation analysis with the brightness temperature (BT) as seen from the Himawari-8 AHI channel 14 (11.2 μm, infrared), which has an original spatial resolution of 2 km at nadir (Bessho et al. 2016).

3) Reanalysis data

ERA5 is the fifth-generation ECMWF atmospheric reanalysis of the global weather, providing estimates of a wide range of atmospheric, land, and oceanic climate variables. The precipitation product includes estimates of precipitation intensity and precipitation classification, generated at 60-min temporal and 0.25° horizontal resolutions (Hersbach et al. 2018, 2020). In this study, we use the mean total precipitation rate (mtpr) and the precipitation classification variables presented in Code Table 4.201 from ECMWF. For comparison purposes, we have regrouped the precipitation classes as follows: liquid (rain, thunderstorm, freezing rain, drizzle, and freezing drizzle), mixed (mixed/ice, wet snow, mixture of rain and snow), and snow (snow, ice pellets, graupel, and hail).

The reanalysis dataset has only directly assimilated radar/rain gauge data over the eastern United States since 2009, leaving the oceanic and most of land precipitation products as model outputs (Hersbach et al. 2020; Lavers et al. 2022). Nevertheless, previous studies have reported that the hydrological cycle is better represented in ERA5 than in ERA-Interim due to improved data assimilation of indirect measurements such as those from microwave and infrared satellite sensors (e.g., GOES, AVHRR, and Himawari-8) and improvements in the model’s microphysical parameterizations (Hersbach et al. 2020). The ERA5 product has been found to produce the most accurate estimates of the mean and seasonal cycle of precipitation over the SO (Boisvert et al. 2020; Manton et al. 2020), but large uncertainties remain in the phase classification and snowfall estimates (Boisvert et al. 2020).

b. Methods

1) Data collocation and averaging

The spatiotemporal geometry and resolution of the OceanRAIN disdrometer, OceanPOL radar, ERA5, and IMERG products are different, confounding any direct comparison, especially when comparing point measurements with area averages (the so called point to area problem) (Loew et al. 2017). To minimize potential errors associated with these issues, we performed a space–time averaging of the disdrometer measurements assuming that the space–time conversion is sufficient to represent the mean precipitation features within the corresponding satellite/reanalysis grid. The hourly mean wind speed measured at the ship was approximately 9.7 m s⁻¹, which corresponds to a distance of about 34 km in an hour. Thus, the 0.25° gridded data are comparable to the hourly averaged shipborne measurements on the horizontal scale. Gridded data (ERA5 and IMERG) were collocated at the ship location using the nearest-neighbor(s) method. If the ship trajectory covered more than one grid point (or parts of them) within an hour, the ERA5 and IMERG data were then averaged over those grid points. To test the sensitivity of the results to the space–time window size, time averages over 30 and 180 min were examined at 0.1° and 0.75° grid resolution. We also tested the collocation method by selecting only the nearest neighbor, as well as a 3 × 3 grid averaging around the mean position of the ship. The results are all found to be largely insensitive to the different spatiotemporal collocation methods used. It should be noted that about 18% (6%) of the ERA5 (IMERG) collocated precipitation rates are below 0.1 mm day⁻¹ [i.e., trace precipitation threshold as defined by the World Meteorological Organization (WMO); Boisvert et al. 2020]. In practice, we are unable to determine the presence and nature of these very light precipitation events suggested by the gridded products, due to the limited measurement sensitivity of the shipborne instruments. Thus, all our analyses are performed for precipitation intensities above the trace precipitation threshold.

2) Spatial analysis

A common shortcoming associated with point-to-area comparisons is the so-called double penalty problem, that is, a mismatch with truth gives both a miss and a false alarm (Keil and Craig 2009). To address this problem, spatial verification metrics are often used to compare precipitation products of different resolutions against a common spatial truth. The fraction skill score (FSS), defined in Eq. (1) (Roberts and Lean 2008), is one of these metrics that directly compare the fractional coverage of grid-mean precipitation (that exceeds a threshold) from two datasets in increasing spatial windows. The FSS returns a value in the range of 0 (no skill) and 1 (perfect match). The useful threshold will determine the minimum useful scale, which is determined by the fractional rainfall over the domain (f, wet area ratio) of the OceanPOL; if f is small enough, the useful threshold is approximately 0.5 (Mittermaier and Roberts 2010).

$\begin{array}{l}\text{FSS}=1-\frac{\frac{1}{N}{\displaystyle \sum _N{(P_m-P_o)}^2}}{\frac{1}{N}\left[{\displaystyle \sum _N{P}_m^2}+{\displaystyle \sum _N{P}_o^2}\right]},\hfill \\ P_o,P_m=\text{precipitation}\text{fraction},o\text{for}\text{OceanPOL},\text{and}m\text{for}\text{ERA5}\text{or}\text{IMERG},\hfill \\ N=\text{number}\text{of}\text{windows},\hfill \\ {\text{FSS}}_{\text{usefull}}=0.5+f/2,\text{where}f\text{is}\text{the}\text{wet-area}\text{ratio}.\hfill \end{array}$(1)

This study evaluated the FSS within a 7 × 7 grid area for the ERA5 and IMERG datasets. This is the maximum number of grid points that fit within the 150-km range of the OceanPOL observations. Additionally, the ERA5, IMERG, and OceanPOL data were compared at 60 min, 0.25° × 0.25° resolution for the estimated “liquid” precipitation events, where the OceanPOL rainfall retrievals are deemed most reliable [see section 2a(1)].

3) Synoptic classification

A synoptic identification is performed using the k-means clustering results produced by Truong et al. (2020) with 2186 soundings collected from four different field campaigns (Fig. 2). The classification identified seven unique clusters across the SO using 15 thermodynamic variables: temperature, relative humidity, zonal wind, meridional wind at 925-, 850-, and 700-hPa levels, respectively, pressure, air temperature, and relative humidity at the surface. Since the sounding data were only available from two of the six voyages included in our study (i.e., CAPRICORN I and II), we have opted to use the “pseudo” soundings from the hourly collocated ERA5 data to classify each hour into one of the seven clusters. Each pseudosounding was assigned to a cluster with the minimum Euclidean distance to one of the seven k-means centroids. For CAPRICORN I and II, our analysis shows that the classification using the ERA5 soundings agrees with the physical sounding classification 89% of the time (not shown). Truong et al. (2022) compared the pseudosoundings and physical soundings and found that ERA5 had an appreciable level of skill in reproducing the clusters over the SO.

View Full Size

A conceptual illustration of the seven thermodynamic clusters derived in Truong et al. (2020). The red circle indicates the warm cluster (W1), the yellow circles indicate the medium clusters (M1–M4), and the dark blue circles indicate the cold clusters (C1–C2). The red (blue) curve with half-circles (triangles) indicates the warm (cold) front. On the right, the main cluster characteristics are also presented. Adapted from Montoya Duque et al. (2022).

Citation: Journal of Applied Meteorology and Climatology 62, 11; 10.1175/JAMC-D-23-0039.1

• Download Figure
• Download figure as PowerPoint slide

3. A comparison between precipitation products: Example cases

Two examples of precipitation events encountered during the voyages are presented to illustrate some of the common precipitation characteristics and potential issues for the comparison.

a. A midlatitude case

In this case, we examined an event when the R/V Investigator was located near 50°S, 140°E between 1200 UTC 16 January and 0000 UTC 20 January 2018. High-level clouds and a low pressure system were located south of the R/V Investigator ahead of the warm sector (Figs. 3a–c). The warm-sector passage was followed by a cold-frontal passage 24 h later (19 January 2018); no cyclone was present within a distance of 5° from the vessel (Fig. 3c).

View Full Size

(from left to right) Warm-air advection in prefrontal areas at 1600 UTC 17 Jan 2018, warm sector at 1000 UTC 18 Jan 2018, and cold front at 1300 UTC 19 Jan 2018, respectively. (top) Himawari-8 BT (K) as colored areas, and contours show the MSLP from ERA5. The orange dot is the R/V Investigator’s location framed in the second- and third-row visual area. Blue (red and pink) dots are cold (warm and stationary) fronts detected with the Berry et al. (2011) scheme. Squares (stars) represent the open (closed) cyclones colored in green (yellow) if they are strong (weak) according to Murray and Simmonds (1991) classification. More details about the front and cyclone detection can also be found in Truong et al. (2020). (second row) ERA5 hourly precipitation rate at 0.25° spatial resolution, with the OceanPOL coverage area highlighted and the disdrometer precipitation estimate indicated at the center. (third row) As in the second row, but for IMERG hourly precipitation rate. (bottom) OceanPOL hourly liquid precipitation rate estimate (mm h⁻¹).

Citation: Journal of Applied Meteorology and Climatology 62, 11; 10.1175/JAMC-D-23-0039.1

• Download Figure
• Download figure as PowerPoint slide

The disdrometer indicated two main precipitation periods at around 1200 UTC 18 January and 1200 UTC 19 January 2018. These events coincide with a local minimum in the mean sea level pressure (MSLP), where the pressure drop is higher during the warm sector than during the last cold-frontal passage. After the warm-sector passage, the ship crosses a cold front and postfrontal conditions, recording no precipitation at the surface, before finally reaching a second (weaker) cold front with precipitation (Fig. 4). Surface air temperature increased during the warm-sector passage, followed by a steep drop in temperature and relative humidity. The Bistatic Radar System for Atmospheric Studies (BASTA) cloud radar detected upper-level clouds prior to the warm-sector passage and some light precipitating clouds during the warm-air advection in prefrontal areas (W1). Low-level precipitating clouds were present during the two cold-frontal passages, and no high-level clouds were visible (Fig. 4e). These patterns are consistent with features shown in the Himawari-8 BT and OceanPOL radar (Figs. 3a–c,j–l). The OceanPOL accumulated precipitation is similar to the ODM470 record, while IMERG collocated precipitation intensities at the ship location are consistently higher than the disdrometer and weather radar estimates (Figs. 4a,b). ERA5 overestimates the accumulated precipitation by about 5 mm (Fig. 4b), although the discrepancies are less pronounced than with IMERG, overestimating the accumulated precipitation by 15 mm and the precipitation intensity by about 1.5 mm h⁻¹ during the passage of the second cold front (Figs. 4a,b).

View Full Size

(a) Time series of precipitation rate (mm h⁻¹) for ERA5 (black), IMERG (gray), ODM470 (blue), and OceanPOL (red); shaded areas indicate the synoptic clusters (Fig. 2). (b) Time series of accumulated precipitation (mm) for the four datasets (lines) and thermodynamic phase when available (scatters). (c) Times series of MSLP (hPa; purple), relative humidity (%; blue), and surface air temperature (°C; red). (d) Time series of BASTA cloud radar reflectivity (dBZ) and ERA5 temperature (contours). Between 1200 UTC 16 Jan and 0000 UTC 20 Jan 2018, the R/V Investigator was located near 50°S, 140°E.

Citation: Journal of Applied Meteorology and Climatology 62, 11; 10.1175/JAMC-D-23-0039.1

• Download Figure
• Download figure as PowerPoint slide

Looking at the spatial structures, ERA5 produced a rainband north of the ship during the prefrontal conditions, around 0100 UTC 18 January 2018, which is spatially consistent with the precipitation pattern observed by OceanPOL. IMERG showed multiple precipitation bands, with the main one located to the south of the ship, which corresponds to the coldest cloud tops depicted in the Himawari-8 BT image (Fig. 3). An hour-by-hour analysis (not shown) indicates that IMERG tended to place precipitation in areas where high-level cloud tops were detected with no or light precipitation at the surface. In the absence of cold BT, IMERG captured the spatial pattern of precipitation, as shown by the OceanPOL, reasonably well, but small-scale precipitation features tended to be missed or underrepresented. In contrast, ERA5 captured the precipitation patterns reasonably well, although it also tended to miss small-scale precipitation cores or have them with lower intensities. Additional analysis (see the supplemental material for a further explanation) suggests that the main source of IMERG precipitation in this case is associated with the PMW-only (48%) algorithm, especially during the periods when IMERG is seen to overestimate the disdrometer values (Fig. 4a). In ERA5, precipitation estimates are primarily the microphysics parameterization (69%).

b. A high-latitude case

In this example, we examined the precipitation structure south of the ocean polar front between 0900 UTC 23 January and 1100 UTC 27 January 2018. The R/V Investigator was located near 58°S, 140°E during that time. This period comprises two different high-latitude cyclone passages (C1) near the R/V Investigator and a trough passage toward the end of the period. The first cyclone is classified as an “open/weak” cyclone by the cyclone detection scheme, with low-level clouds observable in the Himawari-8 imagery (Figs. 5a–c). The second cyclone is diagnosed as a “closed/strong” cyclone, with high-altitude clouds and a warm front approaching the vessel from the north. Toward the end (around 0600 UTC 27 January 2018), high-level clouds associated with the trough conditions (misclassified as a warm front) are present near the vessel, whereas an open/weak cyclone is identified to the east of the ship.

View Full Size

As in Fig. 3, but for two cyclone passages in the high latitudes.

Citation: Journal of Applied Meteorology and Climatology 62, 11; 10.1175/JAMC-D-23-0039.1

• Download Figure
• Download figure as PowerPoint slide

The disdrometer precipitation time series showed three precipitation episodes associated with the two high-latitude cyclones and the trough passage (Fig. 6), with recorded precipitation intensities below 0.7 mm h⁻¹ most of the time. The first C1 episode was characterized by low-level clouds below 2 km, while the second C1 episode (around 1200 UTC 25 January 2018) had a well-defined cloud band present up to 6 km (Fig. 6e). Note that snow and mixed-phase precipitation is likely present in the vicinity of the ship during the second C1 episode, as highlighted in Mace et al. (2023). Clouds up to around 4 km were observed during the third episode under the trough conditions.

View Full Size

As in Fig. 4, but between 0900 UTC 23 Jan and 1100 UTC 27 Jan 2018. The R/V Investigator was located near 58°S, 140°E.

Citation: Journal of Applied Meteorology and Climatology 62, 11; 10.1175/JAMC-D-23-0039.1

• Download Figure
• Download figure as PowerPoint slide

The collocated ERA5 precipitation frequently showed higher precipitation intensities (and accumulated values) than the ODM470 records, while IMERG reported lower intensities throughout most of the period, except around 0400–0600 UTC 27 January 2018, where IMERG overestimated the intensity by about 2 mm h⁻¹ compared to ODM470 (Figs. 6a,b). For the abovementioned period, IMERG and OceanPOL reported similar intensities. However, the BASTA cloud radar reflectivities during this period were similar to/smaller than those associated with the second high-latitude cyclone (Fig. 6e), suggesting that precipitation characteristics were comparable to those during the second cyclone passage. This is consistent with the similar disdrometer precipitation rate estimates in both periods. There is a poor agreement in the precipitation phase between the different datasets. We note that the ambient surface temperature was below 5°C throughout the entire period, where particles may be partially or wholly melted, resulting in large uncertainty in phase classification (and the resulting precipitation intensity estimates) from all datasets.

Spatially, both ERA5 and IMERG reported a rainband to the north of the vessel under the second high-latitude cyclone center conditions (around 1300 UTC 25 January 2018), coincident with high-altitude clouds evident in the cold Himawari-8 BTs (about −53°C). However, the precipitation spatial extent in IMERG seems to be more contracted compared to the ERA5 counterpart, whose spatial precipitation pattern compares better with the OceanPOL radar observations (Fig. 5). Similar spatial features can also be found in an hour-by-hour visual analysis of these products (not shown). With the presence of low BT (around −50°C) from deep clouds at the ship’s location (up to 6 km), IMERG tended to produce heavier precipitation as compared to OceanPOL and the disdrometer estimates. When no deep clouds were present, however, IMERG produced reasonable precipitation intensity estimates but a less accurate spatial extension when compared against OceanPOL, especially during the second high-latitude cyclone passage. The morphing-only algorithm heavily influenced the IMERG precipitation estimates (44%) in this case. During the trough conditions, the highest overestimation in precipitation rate is seen, and the PMW-only is the primary source of estimate. At 1200 UTC 25 January 2018, the precipitation estimates in the rainband are produced by the morphing-only scheme. As in the midlatitude case, ERA5 precipitation is dominated by the microphysics parameterization 68% of the time (see the supplemental material for a further explanation).

Within these two examples, some discrepancies between the OceanRAIN, OceanPOL, ERA5, and IMERG precipitation estimates are noted, for example, IMERG precipitation intensity tended to be higher than ODM470 values during periods of low BTs (high cloud tops); ERA5 precipitation patterns compared better with the OceanPOL observations, even for some of the small-scale precipitation events. Indeed, these discrepancies are repeatedly observed in several other cases we have examined (not shown), which necessitates a more comprehensive analysis where ERA5 and IMERG performance can be more systematically evaluated.

4. Statistical evaluation of the precipitation estimates from six voyages

A total of 4346 h of ODM470 precipitation observations were collected over the six SO voyages conducted between January 2016 and March 2018. Table 2 presents the frequency of occurrence (FoO) and accumulated (accum) precipitation for the total precipitation rate and by the thermodynamic phase of collocated values (rows 1–3). The IMERG product records precipitation 22.3% of the time, roughly equal to that of the ODM470 (25.4%), while ERA5 greatly overestimated the FoO at 64.1%. For the accumulated precipitation, however, ERA5 (436 mm, underestimation of 2.3%) is closer to the observed total (448 mm) than IMERG (615 mm, overestimation of 37%). An overestimation by a factor of 4 is noted for the IMERG mixed-phase precipitation (149 to 39 mm), which is the highest among all phases.

Table 2.

Frequency of occurrence (FoO; %) and accumulation (Accum; mm) of total precipitation and by thermodynamic phase for IMERG, ERA5, and the ODM470 (rows 1–3) for the 4346 h of collocated observations. Concurrent observations of the IMERG and ODM470 (IO) for any precipitation (row 4) and by phase (row 5). Concurrent observations of ERA5 and ODM470 (EO; rows 6 and 7) and all three products (IEO; rows 8 and 9).

View Table

Focusing only on the hours when IMERG and ODM470 observe any precipitation concurrently (IO_precip), we again see that IMERG overestimates the accumulated precipitation 470 to 326 mm), most notably for mixed phase (80 to 17 mm). Examining only the hours when the phase of the precipitation agrees for the IMERG and ODM470 products phase (IO_phase), we limit the analysis to only 9.2% of the 4346 h. Perhaps surprisingly, when mixed phase is observed by both datasets, the ODM470 records a greater accumulation of precipitation (16 to 10 mm), although this is for a very small sample size. The same analysis is made for concurrent ERA5 and ODM470 (EO) observations (rows 6 and 7). When limited to the total precipitation rate, ERA5 underestimates the total accumulation (334 to 441 mm). When matching phase precisely (EO_phase), ERA5 does slightly better in producing the observed accumulated precipitation.

Finally, we look at the most constrained hours when all three instruments observe precipitation (IEO_precip) (row 8) and concurrent phase (IEO_phase) (row 9). About 12% of all records are consistent in total precipitation, and less than 9% have a consistent phase among all three products. During these times (phase consistency), IMERG records twice as much total precipitation as the ODM470 and 10 times more snow/ice. Mixed-phase accumulated precipitation in ERA5 tends to be overestimated by up to 20% during IEO_phase periods compared to the disdrometer. The accumulated total precipitation during the IEO_phase period accounts for the majority (50%) of the accumulated amounts from the three individual datasets (rows 1–3), suggesting that IEO_phase periods may have captured the heavy and widespread precipitation events where key precipitation features are more consistently represented in all products.

Given the relatively poor agreement between the IMERG and ODM470 records, it is worthwhile to examine the contribution of various inputs into the IMERG algorithm at the concurrent times (IO_precip) (see the supplemental material for a further explanation). The IMERG product was never derived from the IR-only technique. Rather, the IMERG algorithm more commonly employed the microwave (PWM only) product (44% only) and the morphing-only product (29%). About 27% of the concurrent hours used the IR technique in combination with other records (mixture), commonly with only a small weight; 75% of these records had a weight below 7%. Sources of error in the concurrent ERA5 precipitation (EO_precip) are more difficult to untangle, with 64% of the ERA5 precipitation being produced by the microphysics and 36% from the convective parameterization. Similar results are found when analyzing IMERG and ERA5 precipitation for all collocated datasets.

A probability distribution function (PDF) of the precipitation rate was calculated for all collocated datasets (Fig. 7). Given the nature of each dataset, we cannot expect a complete match of the PDFs even in the best-case scenario, but any systematic bias will be identifiable. The PDFs suggest that, compared to the ODM470, ERA5 tends to overestimate the FoO of precipitation events with intensities below 0.1 mm h⁻¹, whereas IMERG tends to overestimate the frequency for intensities above 0.5 mm h⁻¹ (Fig. 7a). We further explored the IMERG precipitation rate above 0.5 mm h⁻¹, tracking the input back to mainly the PMW-only algorithm.

View Full Size

PDF of precipitation for IMERG (brown), ERA5 (green), and ODM470 (purple) during six voyages in the SO between January 2016 and March 2018. (a) All thermodynamic phases. (b) As in (a), but only for liquid events detected by the disdrometer while OceanPOL (yellow) was functioning. Dashed lines show frequencies below 1%.

Citation: Journal of Applied Meteorology and Climatology 62, 11; 10.1175/JAMC-D-23-0039.1

• Download Figure
• Download figure as PowerPoint slide

The ODM470 or the OceanPOL detected about 305 h of rain events throughout the 2018 voyages. The OceanPOL indicates a PDF similar to the ODM470, especially for precipitation in the range of 0.05 to 1 mm h⁻¹ (Fig. 7b). Differences outside this range can be associated with the sampling differences, instrument uncertainties in detecting drizzle/light precipitation, and/or possibly the radar retrieval algorithm calibrated for the tropical conditions. Overall, the statistical analysis and the examples presented in section 3 suggest that the OceanPOL and the ODM470 precipitation rate estimates are in reasonable agreement, which justifies the use of the OceanPOL dataset for a spatial analysis presented in section 5.

In summary, the quantitative estimates of snow and mixed-phase precipitation from IMERG and ERA5 are subject to notable errors with differing characteristics. IMERG detected only 20% of the snow and mixed-phase hours compared to the ODM470, while ERA5 detected up to 50% of the hours. For accumulated liquid precipitation, ERA5 seems to agree with the disdrometer records better than IMERG. However, this is largely a result of compensating errors where ERA5 overestimates the frequency and underestimates the intensity of the precipitation. IMERG, on the other hand, tends to overestimate precipitation intensity, which results in a higher accumulated precipitation amount.

5. Spatial analysis: OceanPOL

Bearing in mind the inherent limitations of the OceanPOL dataset as discussed in section 2, the spatial coverage of the C-band radar eliminates the “point to area” problem noted earlier, and the realism of the rain rate retrievals (e.g., Fig. 7) allows the use of this dataset for a spatial verification of the ERA5 and IMERG products. Here, we use a total of 305 liquid precipitation hours collected with the OceanPOL in 2018 to calculate the Fractional Skill Score (FSS) against the ERA5 and IMERG data. For consistency, we have only included precipitation values above the trace threshold (0.1 mm day⁻¹) at 60 min and 0.25° resolution for this analysis. During the selected period, ERA5 produces a maximum rain rate of 6 mm h⁻¹, which is much lower than the maximum rain rate from the IMERG (15 mm h⁻¹) and the OceanPOL (18 mm h⁻¹) records. Rain rates above 6 mm h⁻¹ are present 3% (1%) of the time in IMERG (OceanPOL). ERA5 produces rain rate values above 1 mm h⁻¹ less than 5% of the time, while these values are present 33% and 25% of the time in the IMERG and OceanPOL datasets, respectively (not shown). Given the prevalence of low rain intensities, we calculate the FSS for precipitation thresholds only up to 6 mm h⁻¹. The spatial scale considered in our analysis is also constrained by the spatial range of the OceanPOL observations, which is between 28 and 194 km.

As expected, the FSS results show better skills at larger spatial scales and lower rain rates, except for the very light rain which may not be detectable by the C-band radar, although this does not guarantee that ERA5 and IMERG are correct for these low rain rates (Fig. 8). ERA5 appears to have more skill for smaller spatial scales than IMERG (55 vs 83 km). It is worth noting that the skill of ERA5 increases with the spatial scale (i.e., useful scores are produced for a wider range of precipitation thresholds as the spatial scale increases). The abovementioned tendency is less consistent for IMERG. If we use 0.5 as the useful FSS limit (including light red colors in Fig. 8), IMERG appears to be largely insensitive to the spatial scale for ranges between 55 and 194 km and for thresholds below 1.4 mm h⁻¹, while the ERA5 skillful threshold increases to 3.5 mm h⁻¹. By increasing the precipitation threshold and thereby limiting the skill analysis to more intense precipitation events, our results suggest that ERA5 and IMERG are no longer skillful at any spatial scale for precipitation thresholds above 2.7 mm h⁻¹.

View Full Size

ERA5 and IMERG FSS calculated using the OceanPOL precipitation estimates. (left) OceanPOL vs ERA5 and (right) OceanPOL vs IMERG. All datasets have been averaged to a 60-min, 0.25° × 0.25° resolution. Hashed areas indicate scores below FSS_useful.

Citation: Journal of Applied Meteorology and Climatology 62, 11; 10.1175/JAMC-D-23-0039.1

• Download Figure
• Download figure as PowerPoint slide

In general, ERA5 has better FSS scores than IMERG at large spatial scales for precipitation thresholds between 0.07 and 0.6 mm h⁻¹, which is not particularly surprising given the high frequency of drizzling events present in ERA5. As noted earlier, the overestimation of precipitation intensities in IMERG does not necessarily translate into a better skill in detecting strong precipitation events.

6. Precipitation evaluation by synoptic conditions

It is reported in Montoya Duque et al. (2022) that distinct cloud and precipitation regimes are found to correspond to the seven synoptic clusters over the SO as determined by the k-means clustering from Truong et al. (2020). Therefore, we extend our analysis to examine the skill of ERA5 and IMERG precipitation products under these different weather regimes for the 4346 collocated hours (Table 3). The OceanPOL is excluded in this section as at the time of writing, not enough data are available from the OceanPOL to allow for a statistically robust analysis.

Table 3.

ERA5, IMERG (1 h, 0.25°), and ODM470 disdrometer collocated values as in rows 1–3 in Table 2. Data are differentiated by seven synoptic conditions: warm-air advection in prefrontal areas north of the polar front (W1), high pressure (M1), cold front (M2), postfrontal (M3), warm front (M4), high-latitude cyclone center (C1), and coastal Antarctica (C2). Frequency of occurrence (FoO; %), quartiles 25, 50, and 75 (Q₂₅, Q₅₀, and Q₇₅; mm h⁻¹), and accumulated precipitation (Accum; mm). Hours include nonprecipitating periods.

View Table

Looking first at the IMERG dataset, the precipitation frequency is about half of what was detected by the disdrometer under the cold-frontal (M2) and postfrontal (M3) conditions, although the quartiles of precipitation intensity are more comparable (Table 3). Under W1, IMERG detects a precipitation frequency similar to the disdrometer record, but the upper quartile of precipitation intensity is overestimated, resulting in an overestimate of the accumulated amount (about 200% more). Overall, warm-frontal (M4) precipitation intensity is overestimated by about a factor of 2, but the frequency is underestimated. Precipitation frequency associated with the C1 is similar to the disdrometer record, but the intensity quartiles are overestimated. Unlike the warm-frontal precipitation, these conditions do not translate into an overestimated accumulated precipitation. IMERG detects a small amount of precipitation under the Antarctic coastal condition (C2), although no precipitation was recorded by the disdrometer. We can also note that the C2 and the C1 precipitation are estimated either from the morphing-only or the PMW-only method (see Table S1 in the online supplemental material), as the IR scheme is rarely used when high oblique viewing angles are present (Tan et al. 2019). Warm-frontal precipitation, where IMERG is prone to marked precipitation intensity overestimation, comes mainly from the PMW-only method, consistent with the bias found in the high-latitude case study (section 3b). High pressure (M1) and W1 precipitation estimates are mostly from a mixture of techniques including IR to some extent. M3 precipitation is mostly from PMW-only or morphing-only techniques, and M2 precipitation can come from any of the three analyzed sources.

ERA5 precipitation, on the other hand, tends to overestimate the frequency of precipitation events compared to the disdrometer for all seven clusters. However, the overestimation of frequency tends to almost compensate for the underestimation of precipitation intensity, resulting in similar accumulated amounts. M1 conditions are associated with the greatest underestimation of intensity and overestimation of frequency. ERA5 also tends to record the lowest-intensity events under the Antarctic coastal conditions, while the disdrometer indicates dry conditions. More in-depth analysis from ERA5 would be needed to determine the source of errors.

In summary, our analysis suggests that IMERG tends to overestimate the higher-intensity (upper quartile) precipitation values for all clusters except the M3 conditions. However, during M3 conditions, the precipitation frequency is underestimated. During M4 and C1 conditions, IMERG precipitation intensity overestimation is prevalent. ERA5 overestimates precipitation frequency for all clusters, which leads to an overestimation of accumulated precipitation, even when the intensity is typically lower than the shipborne estimates.

7. Discussion and concluding remarks

Quantitative precipitation estimates over the Southern Ocean (SO) continue to suffer from a high degree of uncertainty, with large differences present among both satellite-based and reanalysis products. This uncertainty arises from the absence of long-term, high-quality observations of precipitation suitable for evaluation across a range of temporal and spatial scales. Recent voyages on the Australian R/V Investigator, however, have provided new observations of precipitation made with both the OceanRAIN maritime disdrometer (ODM470) and a dual-polarization C-band weather radar (OceanPOL).

In this study, we employed these unprecedented observations to evaluate the Global Precipitation Measurement (GPM) Integrated Multi-satellite Retrievals (IMERG) and the ECMWF reanalysis (ERA5) precipitation products at a scale of 60 min and 0.25°, that is, at the mesoscale process level. A summary of key findings is below:

• ERA5 generally overestimates the precipitation frequency compared to the OceanRAIN disdrometer, whereas precipitation intensity is underestimated. This behavior is exacerbated under high pressure conditions.

• IMERG tends to overestimate the precipitation intensity compared to the OceanRAIN disdrometer, especially for warm-frontal and high-latitude cyclone precipitation. Under the postfrontal conditions, IMERG underestimates the precipitation frequency while showing similar precipitation intensities.

• ERA5 has better skill in detecting precipitation events of different spatial scales than IMERG when compared against the OceanPOL C-band weather radar.

• There is little agreement in the phase classification between the OceanRAIN disdrometer, IMERG, and ERA5. The comparisons are complicated by the various assumptions for phase classification in the different datasets.

Previous evaluation of various precipitation products over the SO has largely been limited to MAC. Using a newly developed blended observational dataset, Tansey et al. (2022) found that the annual precipitation frequency is 44% ± 4% at MAC and 27% ± 3% during austral summer. Our analysis across the broader extent of the SO suggests comparable statistics, although the available datasets we employed are somewhat biased toward the warm seasons. More campaign observations spanning different seasons would be desirable to more fully account for potential seasonal cycle in the precipitation features (Lang et al. 2020; Tansey et al. 2022).

Our analysis suggests that ERA5 commonly overestimates the frequency of precipitation and underestimates the intensity of precipitation, although the accumulated precipitation compares reasonably well with the OceanRAIN disdrometer data, with an underestimation of only approximately 2.3%. This is consistent with recent findings in other oceanic regions (Behrangi et al. 2016; Wang et al. 2018; Burdanowitz et al. 2019; Naud et al. 2020). These high-frequency and low-intensity events are commonly referred to as the “drizzle problem,” and are frequently found in global circulation models and often linked to deficiencies in the parameterized convection and/or microphysics. While ERA5 has been suggested to be generally superior to its predecessor, ERA-Interim, our analysis suggests that the long-standing drizzle errors remain prevalent over the SO. These biases have further implications for the modeled cloud cover and latent heat flux, which have a profound influence on the energy budget and atmospheric circulation (Terai et al. 2016; Barrett et al. 2020), particularly over the SO where the model bias in the energy budget is known to be pronounced (Bodas-Salcedo et al. 2014). Encouragingly, our analysis suggests that the ERA5 precipitation intensity estimates are more accurate during heavy and widespread precipitation events over the SO, although these events are less common.

Although previous studies have suggested that IMERG tends to underrepresent the global mean oceanic precipitation as estimated from OceanRAIN (Khan and Maggioni 2019), our analysis shows that IMERG tends to overestimate precipitation intensity over the SO, most notably under the organized synoptic-scale systems such as warm fronts and high-latitude cyclones. This is consistent with some analysis of midlatitude precipitation using rain gauge data over land areas (Xu et al. 2017; Moazami and Najafi 2021) and in moist environments over extratropical oceans (Naud et al. 2018, 2020). Our study found that IMERG underestimates precipitation frequency and intensity under postfrontal (M3) conditions, where shallow convective clouds prevail (Lang et al. 2020, 2021; Montoya Duque et al. 2022). This suggests that the retrieval errors in the IMERG product are likely regime dependent. Also, as suggested by Naud et al. (2020), we find that IMERG has greater challenges in properly detecting frozen precipitation. Our analysis suggests that the IMERG precipitation rate estimates over the SO are more heavily influenced by the PMW-only and the morphing-only schemes, and that the overestimate of precipitation rate is more strongly tied to the PMW-only and the “mixture” methods. While PMW precipitation estimates have been reported to be less accurate over frozen surfaces and in detecting light precipitation (Tan et al. 2019; Naud et al. 2020), tracing back the exact sources of these errors over the SO is beyond the scope of this study, especially considering the 10 PMW satellites and different processing techniques involved (Huffman et al. 2019a,b). We hypothesize that the heavy reliance on the PMW-only technique may help explain the bias of this product under the warm front and high-latitude cyclone conditions over this region, where advection-driven, multilayer clouds are commonly present between 6 and 8 km (Mace et al. 2009; Truong et al. 2020; Montoya Duque et al. 2022). These multilayer clouds, which are not necessarily heavily precipitating, might produce a bulk radiative signature (e.g., brightness temperature) that is similar to the deep convective systems. The morphing scheme is limited by the temporal and spatial resolution since it is based on MERRA-2 (1 h, 2.5°) for computing the motion vectors, which are then interpolated to 30 min and 0.1° (Tan et al. 2019). Also, the morphing method faces challenges in representing short-lived or rapidly moving precipitation systems (Turk et al. 2008). Further, cloud and precipitation microphysics assumptions widely used in the satellite/model algorithms, such as particle size distributions, also need to be revisited for the SO environment, as suggested by Protat et al. (2019a,b), Li et al. (2022).

Both ERA5 and IMERG poorly represent the dry surface conditions near the Antarctic coast (cluster C2), a unique area characterized by dry katabatic winds, which often favors strong evaporation/sublimation of precipitation near the surface (Truong et al. 2020). Despite other factors, the limited resolutions of these products make it difficult to represent the time–space variability of precipitation associated with the complex coastal and orographic processes. Nevertheless, we note that surface precipitation measurements in this environment are also limited. On the other hand, ERA5 and IMERG can perform differently at lower spatial and temporal resolutions as well as in other seasons (Tan et al. 2017; Tansey et al. 2022). We did not include those analyses since in situ data are limited in this region.

In line with findings in Manton et al. (2020), our analysis suggests that ERA5 quantitative precipitation estimates are overall more accurate than those of the IMERG product over the SO. The spatial precipitation patterns are also better captured by ERA5 when evaluated against the (limited) OceanPOL radar data. However, as noted earlier, the scarcity of precipitation measurements over the SO, particularly at high latitudes, remains a significant impediment to the continuous effort of evaluating and improving the precipitation products in this remote region. Enhancing the long-term supports for surface measurement capabilities, such as OceanRAIN and OceanPOL, will continue to be the key to improving our confidence in understanding precipitation as well as a wide range of climate processes.

Data availability statement.

The OceanRAIN disdrometer data used in the study are available at CSIRO via https://doi.org/10.25919/5f688fcc97166 with creative commons attribution 4.0 International. The OceanPOL V2020 weather radar is available on the National Computing Infrastructure (NCI) via https://dx.doi.org/10.25914/5fc4975c7dda8. The CAPRICORN II data used in the case studies are available at CSIRO via https://doi.org/10.25919/5f688fcc97166 with creative commons attribution 4.0 International. ERA5 hourly data (10.24381/cds.bd0915c6) and the IMERG Final Precipitation L3 Half Hourly 0.1° × 0.1° V06 (10.5067/GPM/IMERG/3B-HH/06) datasets were provided by NCI. The NCI downloads and stores ERA5 and IMERG data from ECMWF and NASA/Goddard official downloading websites and makes them available through user registration. The Himawari-8 satellite product was provided by Dr. Alain Protat upon request; the dataset was processed to include high-latitude information during the CAPRICORN field campaign by the Australian Bureau of Meteorology (BoM). All used datasets are available in the in-text data citation references: Huffman et al. (2019c), Hersbach et al. (2018), Protat (2018), UCAR/NCAR EOL (2018), Bessho et al. (2016), Louf and Protat (2020) and Protat and CSIRO (2020).