EPFL | Biomedical Imaging Group | Functional Optimality

On the Functional Optimality of Neural Networks

M. Unser

Proceedings of the XXII Congresso dell’Unione Matematica Italiana (UMI'23), Pisa, Italian Republic, September 4-9, 2023.

Let L be a linear shift-invariant and isotropic operator that is characterized by its radial Fourier profile L^{^}_rad : ℝ → ℝ. We further assume that L^{^}_rad is non-vanishing, except for a zero of order (n₀ − 1) at the origin. This operator is in one-to-one correspondence with the activation function σ_L = ℱ⁻¹{1 ∕ L^{^}_rad} : ℝ → ℝ where ℱ⁻¹ denotes the inverse Fourier transform. We define the corresponding Radon domain regularization operator L_R = K_radRL : ℳ_{L_R}(ℝ^d) → ℳ_even(ℝ ⨉ 𝕊^d−1) where R is the Radon transform, K_rad is the filtering operator of computed tomography (such that R^*K_radR = Id), and ℳ_even is the space of even hyper-spherical bounded measures (see [1] for the precise definition of these elements).

References

M. Unser, "From Kernel Methods to Neural Networks: A Unifying Variational Formulation," arXiv:2206.14625 [cs.LG]

@INPROCEEDINGS(http://bigwww.epfl.ch/publications/unser2302.html,
AUTHOR="Unser, M.",
TITLE="On the Functional Optimality of Neural Networks",
BOOKTITLE="Proceedings of the {XXII} Congresso dell’Unione Matematica
	Italiana ({UMI'23})",
YEAR="2023",
editor="",
volume="",
series="",
pages="",
address="Pisa, Italian Republic",
month="September 4-9,",
organization="",
publisher="",
note="")

© 2023 UMI. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from UMI. This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.