Variational and Stochastic Splines 
Investigators: Sathish Ramani, Shai Tirosh 

Summary: We derive an extended family of smoothingspline estimators for signal denoising and restoration in one and higher dimensions using a variational formulation. We also show that this type of approach is optimal (in the MMSE sense) for the restoration of certain kinds of stochastic processes (including fractallike signals). 

Our basic sampling/reconstruction problem (in one or multiple dimensions) is to recover a continuously defined signal given a series of uniform samples corrupted by additive noise. Depending on the a priori knowledge on the class of signals, there are several alternative ways of approaching this type of problem:
 the variational (or Tikhonov) formulation;
 the stochastic (or Wiener) approach, where the goal is to minimize the meansquare estimation error;
 Bayesian or maximum a posteriori reconstruction.
Interestingly, these formulations all lead to the same kind of solution, which is a generalized smoothing spline. 

Our goal is to extend the concept of a smoothing spline by specifying two types of optimality criteria: variational and stochastic.
 Basic Sampling Problem
In the 1D case, we derived generalized smoothing spline estimators that are associated with a given regularization operator L. We also proposed a stochastic justification by proving that these splines could yield the minimum meansquare error reconstruction of a stationary process whose whitening operator is L.
In essence, we end up with an algorithm that can be viewed as a hybrid version of the Wiener filter where the input is discrete (noisy measurements) and the output is a continuously defined function.
In particular, we considered the estimation of stochastic signals within the Matérn class which leads to the definition of a new brand of splines.
In our most recent work, we worked out a similar stochastic interpretation of the more classical polynomialspline estimators and their fractional extensions. Specifically,
we proved that the fractional smoothingspline estimator with a suitable set of parameters (order of the spline, regularization factor) is statistically optimal in the sense that it provides the minimum meansquare error estimator in the case where the sampled signal is a fractional Brownian motion (fBm) corrupted by additive noise.
Note that these fractal processes, which were introduced by Mandelbrot in 1968, are nonstationary and quite delicate to handle rigorously.
We also investigated a multidimensional version of this problem for the class of images whose power spectrum exhibits a 1⁄f type of decay. We showed that the best linear unbiased estimator is a polyharmonic spline that can be represented as a linear combination of Bsplinelike basis functions. We derived the corresponding multidimensional smoothingspline algorithm and showed that it could be implemented efficiently in the frequency domain. We tested the noisereduction ability of this estimator on a whole variety of natural and biomedical images and observed that it could be tuned to perform essentially as well as the optimal Wiener filter (oracle solution) that uses the full knowledge of the power spectrum of the noiseless signal. Moreover, we found that the optimal smoothing parameters (smoothing parameter and degree of the spline) could be estimated from the measured data.
 Generalized Sampling Problem
We can also make the model more realistic by including the blurring effect of the acquisition device. This too yields a generalized class of splinelike estimators that can be implemented by filtering. The main point is that the optimal solutions (variational or stochastic) are generally not bandlimited.


Collaborations: Prof. Michael Unser, Dr. Thierry Blu, Dr. Dimitri Van De Ville 


Funding: Grants 200020101821 and 200020109415 from the Swiss National Science Foundation 


[5]  S. Ramani, D. Van De Ville, M. Unser, "NonIdeal Sampling and Adapted Reconstruction Using the Stochastic Matérn Model," Best student paper award, Proceedings of the IEEE ThirtyFirst International Conference on Acoustics, Speech, and Signal Processing (ICASSP'06), Toulouse, France, May 1419, 2006, pp. II73II76.

[7]  M. Unser, T. Blu, "A Unifying Spline Formulation for Stochastic Signal Processing [Or How Schoenberg Meets Wiener, with the Help of Tikhonov]," Plenary talk, Second International Conference on Computational Harmonic Analysis, Nineteenth Annual Shanks Lecture (CHA'04), Nashville TN, USA, May 2430, 2004.


