Comparative Study of Training Methods and Architectures of Echo State Networks

Proceedings of the Institute for System Programming of the RAS. 2026. Vol. 38. No. 3. P. 87–114.

Androsov I.

This paper examines echo state networks (ESNs), one of the most prevalent approaches to
implementing reservoir computing. An ESN consists of a recurrent neural network with fixed (untrained)
weights and a readout layer that is typically linear and trainable. This approach enables the creation of energy-
efficient and computationally efficient neural networks capable of real-time learning. However, since ESN
weights are not trained, their selection constitutes a separate challenge that requires careful analysis. The present
paper provides a comparative analysis and review of various ESN architectures and readout layer training
methods. This analysis is based on practical experience with implementations and theoretical foundations,
including studies of how reservoir dynamics depend on the topology of the connectivity graph and the spectrum
of the connectivity matrix. To examine the reservoir structure, tools such as connectivity graph condensation
and linearization of dynamics are utilized, along with the introduction of a novel concept termed graph memory.
In addition to well-established ESN architectures, the review includes less common or previously unapplied
models in the context of reservoir computing, such as reaction-diffusion systems, a single neuron with delay,
FORCE learning, and neural fields. Experimental evaluation is conducted through comprehensive experiments
on the chaotic Mackey-Glass time series prediction task. This paper not only serves as a practical guide for
selecting ESN architectures and readout layer training methods but also identifies promising directions for
future research.

Keywords: дифференциальные уравнения, differential equations, recurrent neural networks, рекуррентные нейронные сети, Reservoir computing, reaction-diffusion systems, нейронные поля, neural fields, резервуарные вычисления, echo state networks, graph memory, сети эхо-состояний, системы реакции-диффузии, графовая память