Learning continuous and piecewise-linear functions and measuring their complexity
Joaquim Campos

Meeting • 29 March 2021

Abstract
In this talk, I will discuss practical applications of the Hessian-Nuclear Total-Variation (HTV) semi-norm. The HTV functional bears a strong resemblance to second-order total variation in 1D. In particular, it also admits a closed-form expression for continuous and piecewise-linear (CPWL) functions and has a similar sparsity-promoting effect. These characteristics motivate us to develop an HTV-regularized learning framework based on a CPWL search space. In this manner, the infinite-dimensional learning problem can be exactly recast as a finite-dimensional one, which can be efficiently solved. Through numerical examples, we show that our algorithm constructs CPWL models with few facets. In the second part, I will briefly discuss the ongoing project on the use of the HTV to measure the complexity of CPWL models, with special focus on ReLU networks.