Optimal Estimation Retrievals and Their Uncertainties: What Every Atmospheric Scientist Should Know

Maximilian Maahn Cooperative Institute for Research in Environmental Sciences, University of Colorado Boulder, and NOAA/Physical Sciences Lab, Boulder, Colorado

Search for other papers by Maximilian Maahn in
Current site
Google Scholar
PubMed
Close
,
David D. Turner NOAA/Global Systems Lab, Boulder, Colorado

Search for other papers by David D. Turner in
Current site
Google Scholar
PubMed
Close
,
Ulrich Löhnert Institute for Geophysics and Meteorology, University of Cologne, Cologne, Germany

Search for other papers by Ulrich Löhnert in
Current site
Google Scholar
PubMed
Close
,
Derek J. Posselt Jet Propulsion Laboratory, California Institute of Technology, Pasadena, California

Search for other papers by Derek J. Posselt in
Current site
Google Scholar
PubMed
Close
,
Kerstin Ebell Institute for Geophysics and Meteorology, University of Cologne, Cologne, Germany

Search for other papers by Kerstin Ebell in
Current site
Google Scholar
PubMed
Close
,
Gerald G. Mace Department of Atmospheric Science, University of Utah, Salt Lake City, Utah

Search for other papers by Gerald G. Mace in
Current site
Google Scholar
PubMed
Close
, and
Jennifer M. Comstock Pacific Northwest National Laboratory, Richland, Washington

Search for other papers by Jennifer M. Comstock in
Current site
Google Scholar
PubMed
Close
Free access

Abstract

Remote sensing instruments are heavily used to provide observations for both the operational and research communities. These sensors do not provide direct observations of the desired atmospheric variables, but instead, retrieval algorithms are necessary to convert the indirect observations into the variable of interest. It is critical to be aware of the underlying assumptions made by many retrieval algorithms, including that the retrieval problem is often ill posed and that there are various sources of uncertainty that need to be treated properly. In short, the retrieval challenge is to invert a set of noisy observations to obtain estimates of atmospheric quantities. The problem is often complicated by imperfect forward models, by imperfect prior knowledge, and by the existence of nonunique solutions. Optimal estimation (OE) is a widely used physical retrieval method that combines measurements, prior information, and the corresponding uncertainties based on Bayes’s theorem to find an optimal solution for the atmospheric state. Furthermore, OE also allows the relative contributions of the different sources of error to the uncertainty in the final retrieved atmospheric state to be understood. Here, we provide a novel Python library to illustrate the use of OE for inverse problems in the atmospheric sciences. We introduce two example problems: how to retrieve drop size distribution parameters from radar observations and how to retrieve the temperature profile from ground-based microwave sensors. Using these examples, we discuss common pitfalls, how the various error sources impact the retrieval, and how the quality of the retrieval results can be quantified.

Supplemental material on Python code belonging to all shown retrieval examples available at https://github.com/maahn/pyOptimalEstimation_examples

Corresponding author: Maximilian Maahn, maximilian.maahn@uni-leipzig.de

Abstract

Remote sensing instruments are heavily used to provide observations for both the operational and research communities. These sensors do not provide direct observations of the desired atmospheric variables, but instead, retrieval algorithms are necessary to convert the indirect observations into the variable of interest. It is critical to be aware of the underlying assumptions made by many retrieval algorithms, including that the retrieval problem is often ill posed and that there are various sources of uncertainty that need to be treated properly. In short, the retrieval challenge is to invert a set of noisy observations to obtain estimates of atmospheric quantities. The problem is often complicated by imperfect forward models, by imperfect prior knowledge, and by the existence of nonunique solutions. Optimal estimation (OE) is a widely used physical retrieval method that combines measurements, prior information, and the corresponding uncertainties based on Bayes’s theorem to find an optimal solution for the atmospheric state. Furthermore, OE also allows the relative contributions of the different sources of error to the uncertainty in the final retrieved atmospheric state to be understood. Here, we provide a novel Python library to illustrate the use of OE for inverse problems in the atmospheric sciences. We introduce two example problems: how to retrieve drop size distribution parameters from radar observations and how to retrieve the temperature profile from ground-based microwave sensors. Using these examples, we discuss common pitfalls, how the various error sources impact the retrieval, and how the quality of the retrieval results can be quantified.

Supplemental material on Python code belonging to all shown retrieval examples available at https://github.com/maahn/pyOptimalEstimation_examples

Corresponding author: Maximilian Maahn, maximilian.maahn@uni-leipzig.de
Save