Chaotic itinerancy in reservoir computing

Recently, the paradigm of reservoir computing (RC) has attracted attention as a new way of recurrent neural network (RNN) training [1]. Especially, Sussillo and Abbott proposed a version of RC, called FORCE-learning[2] and they showed how chaotic activity in a RNN is useful for generating various temporal patterns.
In this study, we construct a machine based on the FORCE-learning scheme for generating a set of temporal patterns that are selectable by the combination of trigger pulses through multiple channels.
We show that this machine can be actually realized, depending on the adequate choice of parameters of the reservoir. When the machine shows optimal performances, it also shows intermittent transitions among several typical patterns without any trigger inputs, i.e., spontaneous mode, which reminds us chaotic itinerancy [3]. We characterize this pliability of chaos in terms of the large deviation fluctuations of Lyapunov exponents and visualize the itinerant dynamics using the manifold learning such as t-SNE.

[1] H. Jäger, W. Maas and J. Principe, Neural Networks 20, 287 (2007).
[2] D. Sussillo and L.F. Abbott, Neuron 63, 544 (2009).
[3] K. Kaneko and I. Tsuda. Chaos 13, 926 (2003).