EPFL | Biomedical Imaging Group

Inference for Generalized Linear Models via Alternating Directions and Bethe Free Energy Minimization

S. Rangan, A.K. Fletcher, P. Schniter, U.S. Kamilov

Proceedings of the 2015 IEEE International Symposium on Information Theory (ISIT'15), Hong Kong, People's Republic of China, June 14-19, 2015, pp. 1640–1644.

Generalized Linear Models (GLMs), where a random vector x is observed through a noisy, possibly nonlinear, function of a linear transform z = A x arise in a range of applications in nonlinear filtering and regression. Approximate Message Passing (AMP) methods, based on loopy belief propagation, are a promising class of approaches for approximate inference in these models. AMP methods are computationally simple, general, and admit precise analyses with testable conditions for optimality for large i.i.d. transforms A. However, the algorithms can easily diverge for general transforms. This paper presents a convergent approach to the generalized AMP (GAMP) algorithm based on direct minimization of a large-system limit approximation of the Bethe Free Energy (LSL-BFE). The proposed method uses a double-loop procedure, where the outer loop successively linearizes the LSL-BFE and the inner loop minimizes the linearized LSL-BFE using the Alternating Direction Method of Multipliers (ADMM). The proposed method, called ADMM-GAMP, is similar in structure to the original GAMP method, but with an additional least-squares minimization. It is shown that for strictly convex, smooth penalties, ADMM-GAMP is guaranteed to converge to a local minima of the LSL-BFE, thus providing a convergent alternative to GAMP that is stable under arbitrary transforms. Simulations are also presented that demonstrate the robustness of the method for non-convex penalties as well.

@INPROCEEDINGS(http://bigwww.epfl.ch/publications/rangan1501.html,
AUTHOR="Rangan, S. and Fletcher, A.K. and Schniter, P. and Kamilov,
	U.S.",
TITLE="Inference for Generalized Linear Models via Alternating
	Directions and {B}ethe Free Energy Minimization",
BOOKTITLE="Proceedings of the 2015 IEEE International Symposium on
	Information Theory ({ISIT'15})",
YEAR="2015",
editor="",
volume="",
series="",
pages="1640--1644",
address="Hong Kong, People's Republic of China",
month="June 14-19,",
organization="",
publisher="",
note="")

© 2015 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from IEEE. This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.