EPFL | Biomedical Imaging Group | Reservoir Computing

Asymptotic Stability in Reservoir Computing

J. Dong, E. Börve, M. Rafayelyan, M. Unser

Proceedings of the 2022 International Joint Conference on Neural Networks (IJCNN'22), Padua, Italian Republic, July 18-23, 2022, 8 p.

Reservoir Computing is a class of Recurrent Neural Networks with internal weights fixed at random. Stability relates to the sensitivity of the network state to perturbations. It is an important property in Reservoir Computing as it directly impacts performance. In practice, it is desirable to stay in a stable regime, where the effect of perturbations does not explode exponentially, but also close to the chaotic frontier where reservoir dynamics are rich. Open questions remain today regarding input regularization and discontinuous activation functions. In this work, we use the recurrent kernel limit to draw new insights on stability in reservoir computing. This limit corresponds to large reservoir sizes, and it already becomes relevant for reservoirs with a few hundred neurons. We obtain a quantitative characterization of the frontier between stability and chaos, which can greatly benefit hyperparameter tuning. In a broader sense, our results contribute to understanding the complex dynamics of Recurrent Neural Networks.

@INPROCEEDINGS(http://bigwww.epfl.ch/publications/dong2201.html,
AUTHOR="Dong, J. and B{\"{o}}rve, E. and Rafayelyan, M. and Unser, M.",
TITLE="Asymptotic Stability in Reservoir Computing",
BOOKTITLE="Proceedings of the 2022 International Joint Conference on
	Neural Networks ({IJCNN'22})",
YEAR="2022",
editor="",
volume="",
series="",
pages="",
address="Padua, Italian Republic",
month="July 18-23,",
organization="",
publisher="",
note="")

© 2022 IEEE. Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from IEEE. This material is presented to ensure timely dissemination of scholarly and technical work. Copyright and all rights therein are retained by authors or by other copyright holders. All persons copying this information are expected to adhere to the terms and constraints invoked by each author's copyright. In most cases, these works may not be reposted without the explicit permission of the copyright holder.