1. Introduction

As a result of better numerical weather prediction (NWP) models, more powerful computers, new satellite observations, and more efficient and effective data assimilation systems, the forecast skill of midtropospheric synoptic flow patterns has steadily improved over the past few decades. Today’s 4-day forecasts of those patterns are as accurate as 3-day predictions were just a decade ago and as 2-day forecasts were 2 decades ago. Forecasts for the Southern Hemisphere, where satellites provide the bulk of observations, are now almost as accurate as those for the Northern Hemisphere.

However, the progress in forecasting weather elements that are of particular public interest, such as clouds, quantitative precipitation, and precipitation type, has been less dramatic. To date, the assimilation of satellite measurements has focused on the clear atmosphere. But satellite observations in the visible, infrared, and microwave provide a great deal of information on clouds and precipitation. The issue is how to use this information to improve the initialization of clouds and precipitation in models. Since clouds and precipitation often occur in sensitive regions for forecast impacts, such improvements are likely necessary for continuing to acquire significant gains in weather forecasting.

To accelerate progress in the field, the Joint Center for Satellite Data Assimilation (JCSDA), a joint activity of the National Oceanic and Atmospheric Administration (NOAA), National Aeronautics and Space Administration (NASA), and Department of Defense (DoD), sponsored an international workshop in May 2005. Participants included experts in the multiple scientific disciplines involved—satellite observations of clouds and precipitation, radiative transfer (RT), modeling clouds and precipitation in NWP, and data assimilation.

2. Issues concerning observations

Starting with the Tropical Rainfall Measuring Mission (TRMM), passive instruments are now complemented with active sensors (radars and lidars) providing information on the vertical distribution of clouds and precipitation. The recently launched CloudSat carries a cloud radar that measures vertical profiles of cloud water and ice and vertical cloud boundaries with a 250-m resolution. The Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) satellite has a 2-wavelength lidar that measures ice and water extinction profiles, and cloud heights of optically thin clouds, with a vertical resolution of 30–60 m. Detailed observations of cloud properties are made at the three Atmospheric Radiation Measurement Program (ARM) sites of the U.S. Department of Energy and also from the Cloudnet European initiative. Although all these observations have limited global coverage during any (3–12 h) data assimilation period, they will be useful for calibrating and validating retrievals from passive sensors and for measuring and validating the modeling of cloud properties and the distribution of clouds in the vertical.

Ground-based radar and rain gauge observations provide some validation for satellite precipitation measurements. The former suffer from an inexact relationship between radar reflectivity and precipitation rate. The latter are associated with large representativeness errors. There is little or no ground truth over oceans, although progress is being made in developing acoustic rain gauges. While most ground-based measurements of precipitation may be considered more accurate than retrievals based on satellite data, a more global, continuous, and consistent observation of cloud and precipitation properties represents a fundamental requirement of NWP.

Polar orbiting satellites provide poor temporal sampling (6-h in the worst case in the Tropics) relative to the time scales of precipitation development. Geostationary satellite infrared (IR) measurements are hampered by the poor relationships between cloud-top temperatures and the underlying cloud and precipitation physics. Microwave (MW) measurements are affected by sensitivity to the highly variable land surface emissivity and similar optical properties of cloud water and light rainfall that limit the detectability and retrieval accuracy of either component. Current observations also lack sensitivity to drizzle and snowfall.

Specific workshop recommendations regarding observations include 1) expanding the use of ARM site observations and conducting well-planned field campaigns to provide better validation of satellite cloud and precipitation measurements; 2) designing validation programs, such as those for the Global Precipitation Measurement (GPM) mission, but with data assimilation applications in mind; and 3) exploiting millimeter-wave sounding channels [e.g., Advanced Microwave Sounding Unit-B, Special Sensor Microwave Imager (SSM/I)] with improved sensitivity to snow and drizzle to retrieve variables describing clouds and precipitation.

3. Issues concerning models

The diverse set of models and associated components required for data assimilation have varying degrees of reliability. Atmospheric dynamics at horizontal scales larger than 100 km or so are typically handled quite well both in terms of analysis and short-term forecast skill. Moreover, operational NWP models are able to predict the location in space and time of clouds associated with large-scale organized systems, but their skill degrades as the strength of synoptic forcing or the degree of larger-scale organization decreases.

Large uncertainties remain in many of the physical parameterizations used in both forecast models and in relating observations to analysis fields. Modeling of some diabatic processes, particularly those associated with moist convection and the radiative effects of clouds, are still unreliable. Spatial distributions of clouds, particularly over warmer oceans, can vary substantially when different combinations of well-tested physical parameterizations are used. Even in regions where strong dynamics tend to generate significant precipitation and clouds that are well delineated, quantitative reliability remains lacking. Arguably the biggest issue within NWP remains the proper representation of subgrid-scale precipitating convection within coarser-resolution models. Other schemes needing improvements include the parameterization of the effects of shallow convective clouds associated with the planetary boundary layer, microphysics of cloud formation (especially the initiation of ice-phase processes), exchanges between the atmosphere and the surface (this includes the myriad of land surface processes and increasingly complicated coupling with the ocean), and overlap of multiple cloud layers for radiative calculations. Finally, improving the coupling between physical processes by exchanging more information among physical parameterizations is becoming just as important, complicating both the model software and interpretations of results.

Even schemes apparently based on fundamental physics, such as “explicit microphysical precipitation schemes,” are in fact highly parameterized with much accompanying uncertainty. Also, details such as ice cloud particle shape and size distributions, which are critical for estimating radiative effects, are rather crudely represented even in these schemes. The assimilation of satellite radiances in cloudy regions necessitates that the model counterparts are reasonably well described. As a consequence, moist physical processes in NWP models need to have enough realism, including information on hydrometeor properties to provide reasonably good representations of observed radiances. Useful, linearized versions of these models are needed for variational data assimilation schemes (to efficiently solve the optimization problem).

Specific workshop recommendations for improving the utility of the various models required for data assimilation include 1) constructing several robust observational or simulated cloud-resolving model datasets for validating process models and components, such as those used for validating single-column models; 2) developing moist convective schemes that are more compatible with data assimilation applications, requiring the prediction of hydrometeor characteristics that are directly related to observations (e.g., particle characteristics that govern radiative scattering) and validation methods that use additional metrics (e.g., not just time-mean surface precipitation rates); and 3) developing collaborative programs involving the diverse subdisciplines within the atmospheric community having the expertise required for successful cloud and precipitation assimilation.

4. Issues concerning radiative transfer

The development of fast, accurate IR and MW radiative transfer models for clear atmospheric conditions has enabled the direct assimilation of clear-sky radiances in NWP models. Recently, various models have been extended to also handle the scattering and emission processes that dominate cloud and precipitation RT. Linearized versions of these models, which are crucial components for variational data assimilation, have also been developed. The Joint Center for Satellite Data Assimilation has developed a community RT model (CRTM) framework that will allow for the faster implementation of new RT algorithms into operations. A similar RT code (RTTOV-8) has been made available to the NWP community through the NWP Satellite Applications Facility initiated in Europe by the European Organisation for the Exploitation of Meteorological Satellites (EUMETSAT).

RT models currently suffer from several difficulties and limitations. Emission and scattering by precipitation and clouds are functions of particle properties (e.g., shape, particle size distributions, etc.) that are not currently predicted or diagnosed in NWP models. Accounting for the spatial inhomogeneity of clouds and precipitation can require complex, time consuming techniques for computing scattering. This is particularly important for the assimilation of high-frequency MW channels. Uncertainties in surface radiative property modeling (e.g., emissivity and reflectivity) remain large in many geographical regions, particularly over land and ice. There are few comprehensive datasets for fully assessing the accuracies and performances of RT models.

Progress in modeling RT in clouds and precipitation can be accelerated by implementing the following workshop recommendations: 1) constructing sets of high-quality satellite and associated in situ observations, the latter including condensate sizes and shapes, to fully assess RT model performance; 2) characterizing the biases and standard deviations of simulated radiances; 3) determining the mean particle size from satellite visible/near-infrared and/or microwave observations; 4) using improved moist physical schemes (considering more detailed particle size distribution information) in the forward model calculations of operational NWP models; and 5) developing and improving fast radiative transfer schemes for clouds and precipitation needed for complete 3D or 4D data assimilation systems that include many observation types.

5. Issues concerning assimilation

Precipitation estimated from radar data or derived from IR and MW satellite radiances has been assimilated in some operational or research models for many years. The assimilation techniques began with empirical nudging and more recently have evolved toward optimal estimation theory. Variational techniques, including both 3D and 4D schemes, are currently employed operationally at a few centers. These include the Japan Meteorological Agency (using a regional model with ground-based radar observations), NOAA’s National Weather Service (using a global model with satellite observations), and the European Centre for Medium-Range Weather Forecasts (currently the only operational, global, 4D variational assimilation system using satellite precipitation observations). Depending on the design of the actual data assimilation system, either the assimilation of derived products or radiances/reflectivities is preferred. Success according to some measures has been sufficient to continue the operational practices. Forecast improvements thus far have been generally limited, however, to the first day and for some systems to only the first 6 h.

There are several impediments to assimilating cloud and precipitation observations in current data assimilation systems. The range of space scales is broad (microphysical to planetary), as is the range of time scales, although most current systems have been principally designed and validated for the assimilation of synoptic-scale features. Precipitation and cloud formation depend strongly on aspects of the atmospheric flow that are less well observed and modeled than conventional fields. Many fundamental characteristics of the observations, such as their mismatch with model grid representations, render them difficult to assimilate. This includes current inaccuracies of their corresponding observation operators (e.g., RT and moist physics models). Critical dynamic balance constraints currently incorporated in data assimilation schemes may require redesign in order to match observations in regions of strong latent heat release.

The nonlinearity of moist physical processes implies that assimilation schemes based on linear theory should be modified to accommodate these new observations. Various studies have shown that these errors can be large and nonGaussian. There is currently little theoretical guidance to modify current schemes. Usual assumptions of unbiased models and assumptions of a perfect trajectory model (in the 4D context) must be relaxed.

Besides inherent difficulties, progress in assimilating these data has been slowed due to insufficient examination of several fundamentals. While many implications of chaotic dynamics and physics have been explored since the advent of numerical weather prediction, others have yet to be considered. Particularly, knowledge of any limits in the predictability of clouds and precipitation (and their scale dependency) would help determine both appropriate goals and paths to attain them. More complete and critical assessments of past and current attempts to assimilate these data would help to identify the importance of neglected issues and provide a firmer foundation for future developments. The consistent use of common sets of metrics applied to precipitation and cloud forecast errors, analysis increments, and analysis residuals would help to establish a basis of information to measure future improvements as well as insight regarding sources of error.

Although the assimilation problem integrates many issues, light can be shed on many aspects of the problem by isolating them. This requires sufficient understanding of the full problem so that the isolated context remains appropriate. Included are examinations of the Jacobians of physical parameterization schemes and forward RT models, adjoint-based sensitivity analysis, predictability characterization, and better inclusion of statistical considerations in model development, including appropriate stochastic modeling. These problems are ideally suited for traditional university research where the function of a large, complete data assimilation system is unnecessary.

Workshop recommendations to accelerate progress include 1) comparing background-derived (i.e., model simulated) estimates with actual observations for data concerning precipitation or cloud properties in order to provide baseline estimates of information content and early warnings of potential problems; 2) entraining model developers in designing physical parameterizations schemes for data assimilation applications (with possible requirements of linearization, regularization, simplification, or stochastic modeling); 3) encouraging both data and model providers to pay particular attention to estimating characteristics of the errors in their products or formulations; 4) developing prototype precipitation and cloud data assimilation systems even if their impact on forecast skill (using standard measures) is initially neutral; 5) developing new forecast skill measures to evaluate the performance of NWP models regarding the presence of clouds and precipitation and their effects on other fields; and 6) conducting well-designed and carefully interpreted predictability experiments to determine what increase of precipitation and cloud forecast skill is a realistic target and what specific kinds of improvements are required to attain that skill.

6. Overarching recommendation

The assimilation of observations related to clouds and precipitation requires combined efforts among the observation, modelling, and data assimilation communities. It is truly interdisciplinary, requiring attention to many details of each aspect of the problem. Although the NWP community has attempted to develop much of the required expertise within itself, progress would be accelerated by entraining collaborators willing to familiarize themselves with the details of the assimilation problem. Some problems can be posed simply and independently enough to be investigated by individual researchers or graduate students. Others require more concerted efforts and greater levels of expertise. Communication is currently difficult, with concepts considered basic within one community being foreign to another. The workshop encourages the Joint Center for Satellite Data Assimilation, the major operational NWP Centers, and appropriate funding agencies to enhance opportunities for communication and to support new and stronger collaborations.

7. Workshop presentations

All workshop presentations are posted on the Joint Center for Satellite Data Assimilation Web site (see http://www.jcsda.noaa.gov/CloudPrecipWkShop/index.html).