Bayesian Inference for Inverse Problems: From Sparsity-Based Methods to Deep Neural Networks
M. Unser, P. Bohra
Tutorial, Twentieth IEEE International Symposium on Biomedical Imaging (ISBI'23), Cartagena de Indias, Republic of Colombia, April 18-21, 2023.
Inverse problems are central to biomedical imaging, examples being deconvolution microscopy, computed tomography, magnetic resonance imaging, or optical diffraction tomography. This tutorial is centered on Bayesian inference, which is a powerful method for the resolution of such problems. Our goal is to present different flavors of this approach, from classical methods to more recent deep-learning-based methods, in a concise and digestible way.
The tutorial is divided into two parts: model-based approaches and learning-based approaches. We begin the tutorial by briefly describing variational approaches that are in common use for the resolution of ill-posed problems. In particular, we discuss Tikhonov regularization and the transition to more sophisticated (and better-performing) sparsity-based regularizers (TV denoising, wavelet shrinkage). We then introduce the Bayesian formulation of inverse problems in a general setting and we highlight the potential of such an approach. We detail the different components of the framework such as the likelihood function, the prior distribution, the posterior distribution (the main quantity of interest), and the different ways one can use the posterior distribution to perform inference. Next, we dive into the world of stochastic signal models for the specification of the prior distribution. We look at the classical Gaussian processes and their non-Gaussian counterparts which admit a sparse expansion in wavelet-like bases and are thus termed sparse processes. We then focus on the maximum a posteriori (MAP) estimators which we show are compatible with the commonly used variational techniques that we described in the beginning of the tutorial. We outline some optimization algorithms (e.g., forward-backward splitting, alternating direction method of multipliers) that compute such estimators. We also illustrate their use on image-reconstruction tasks such as deconvolution and computed tomography. Finally, we talk about the notion of sampling from the posterior distribution which allows one to perform more advanced Bayesian inference as compared to MAP estimation. We give a short primer on Markov chain Monte Carlo methods as they enable efficient posterior sampling. To make things clearer, we provide two concrete applications. First, we derive sampling schemes to compute the minimum mean-square error estimators for sparse stochastic processes (SSPs) which we use to develop a statistical benchmarking framework for signal-reconstruction algorithms. Then, we look at an example of uncertainty quantification in simple imaging tasks (e.g., deblurring).
We begin the second part of the tutorial by discussing the advent of deep-learning-based methods for the solution of inverse problems (the learning revolution). Specifically, we review the first two generations of neural-network-based methods which can be viewed as the learned counterparts of MAP estimators (Tikhonov regularization and sparsity-promoting techniques). We then mention pitfalls associated with such methods, which are especially relevant in sensitive applications such as biomedical imaging. We also present benchmarking results for CNNs based on the statistical framework for SSPs described in the first part of the tutorial. Next, we move on to the development of posterior-sampling schemes that involve neural-network-based priors. We focus on two kinds of learned priors: implicit priors defined through denoising CNNs and deep generative priors that involve variational autoencoders, generative adversarial networks, and score-based diffusion models. We detail efficient sampling schemes for many of them. We illustrate the power of one such GAN-based method by looking at nonlinear inverse problems such as phase retrieval and optical diffraction tomography.
@INPROCEEDINGS(http://bigwww.epfl.ch/publications/unser2303.html,
AUTHOR="Unser, M. and Bohra, P.",
TITLE="Bayesian Inference for Inverse Problems: {F}rom Sparsity-Based
Methods to Deep Neural Networks",
BOOKTITLE="Twentieth IEEE International Symposium on Biomedical Imaging
({ISBI'23})",
YEAR="2023",
editor="",
volume="",
series="",
pages="",
address="Cartagena de Indias, Republic of Colombia",
month="April 18-21,",
organization="",
publisher="",
note="Tutorial")