Splines for Image Processing
Spline methods for the continuous/discrete processing of images
Principal Investigator: Prof. Michael Unser
Participants: Shayan Aziznejad, Pakshal Borha, Joaquim Campos, Thomas Debarre, Stanislas Ducotterd, Alexis Goujon, Yan Liu, Rahul Parhi, Mehrsa Pourya
Period: October 1, 2018 - September 30, 2023.
SNSF Grant: 200020_184646
Abstract
Our aim is to develop a comprehensive framework to address a whole class of problems that are usually formulated in the continuous domain, but call for a digital implementation. Our practical focus is on image processing while the theoretical context is provided by splines. This project started in 1998 and its scope has kept on broadening and becoming more mathematical as more fundamental aspects of splines (in relation to differential equations, wavelets, linear systems, estimation and regularization theory, and stochastic processes) are being uncovered. The specific spline-related research topics that will be investigated during this term are:
A. Splines and operators: Beyond the theory of RKHS
Splines can be specified as solutions of variational problems with a quadratic regularization that involves a suitable differential operator L and a native space that is a reproducing kernel Hilbert space (RKHS). We shall generalize the concept by switching to generalized total-variation (gTV) regularization and by enlarging the class of admissible L through the consideration of operators that are not necessarily shift-invariant. The critical point will be to specify the corresponding native space under the constraint of universality (i.e., the ability to represent any continuous function with an arbitrary degree of precision). We are aiming at an extended Banach-space framework that encompasses RKHS but supports the use of more general kernel-type expansions where the kernels are not necessarily positive-definite nor located on the data points.
B. Multi-splines
Cardinal splines are representable using basis functions that are shifted replicates of a single generator. The extension that we propose in this grant is to consider multiple generators. First, we shall study multi-splines that are direct sums of conventional splines. The challenge there is to construct the shortest basis functions. This will help us to specify a multi-spline calculus for signal processing and develop generalized-sampling techniques. In particular, we shall investigate the use of Hermite-splines for image processing. Second, we shall consider multi-component splines to let us extend the classical techniques to the vectorial setting. The investigated topics will include the construction of vector-valued B-splines, the solution of variational vector-valued problems, and the connection with vector-valued stochastic processes.
C. Splines and machine learning
The intimate link between splines and RKHS connects them to kernel methods in supervised learning. Our intent is to draw similar connections with deep learning via the introduction of suitable forms of gTV regularization. We shall consider the problem of the optimization of individual activation functions in a deep neural network. We shall investigate the properties of deep-spline architectures where the activation functions are adaptive splines and propose effective training schemes. We shall derive representer theorems, including vector-valued generalizations, for sparsity-promoting regularization functionals.
D. Splines and inverse problems
We have shown recently that any linear inverse problem with gTV regularization admits a global solution that is a nonuniform spline whose type is matched to the regularization operator. The difficulty is that the underlying spline is adaptive, meaning that the determination of the location of its knots is also part of the optimization problem. Our aim is to develop efficient numerical schemes for finding the continuous-domain solutions of such problems with an arbitrary degree of precision. We shall also consider an extended formulation that involves multiple regularization operators. The methods will be applied to the deconvolution of signals and to the reconstruction of magnetic-resonance images.
E. Splines and stochastic processes
We shall characterize the complete (non-Gaussian) family of self-similar processes as solutions of stochastic differential equations that involve derivatives of complex order. We shall exploit the connection between splines and sparse stochastic processes to develop new ways of generating non-Gaussian processes. Finally, we shall propose an extended functional framework for the MMSE reconstruction of (not necessarily stationary) Gaussian processes from linear measurements, which we postulate to yield a kind of ``generalized'' splines.
This research should produce: (1) a theoretical impact in the fields of image processing and applied mathematics, due to the novelty and unifying character of the formulation; (2) a practical impact through efficient algorithms and novel computational solutions. The developments in B and D are motivated by concrete problems in bioimaging. The end product will be open-source software that will be made freely available to the biomedical research community.