Representation of Stable AR(1) Processes with Partially Coupled Coefficients in Transform Domain

Mohamad Dia

Semester Project

Section of Communication Systems, EPFL

July 2013

Abstract

n this work, we focus on the Independent Component analysis of sparse AR(1) processes driven by symmetric-alpha-stable noise in order to produce its maximally independent representation in the transform domain. We consider the minimization problem of the dependency measured by the Kullback-Leiber divergence between the joint pdf of the original signal and the product of the marginals in the transformed domain. For the model-based orthogonal ICA, we compare different optimization algorithms, and we propose an efficient algorithm based on the Gradient Descent with adaptive rate and momentum learning. The proposed algorithm outperforms – in terms of convergence rate – the classical GD algorithm and even other numerical optimization methods such as Newton’s Method. We show that the wavelet basis expansion (operator-like wavelets transforms including the Haar transform) is very close to the optimal orthogonal ICA solution especially for the sparse case (and can coincide with optimal solution for the very sparse case). After showing that our measure of dependency captured by the KLD can be estimated based on large number of samples, the work was then extended to the non-orthogonal case and applied on the sample-based ICA in order to estimate the system model. The results show that the solution of non-orthogonal ICA tends to approximate the system model for a special case of the process.