Gated recurrent neural networks as a framework for studying neuromodulation: improving memory capacity and appearance of generalization due to neuropeptide-like action

Vanilla recurrent neural networks (RNNs) lack the ability to handle long term temporal dependencies and time warping in the input data. This can be alleviated by multiplicative gating of the synaptic layer. Meanwhile, synaptic firing in living organisms is modulated through molecules, i.e. neuropeptides, that can communicate neurons many synapses away from each other. Here we study the effects of multiplicative, neuropeptide-like gating on associative memory retrieval using a gated RNN. Using numerical simulations and dynamical mean field theory, we find that gating enhances the memory capacity of a graded time continuous Hopfield model and possibly gives rise to generalization by the appearance of continuous attractors around the fixed-point patterns. Furthermore, our model could also serve to infer neuropeptide connectivity for a behavior by inferring from neural firing data exhibiting attractor dynamics such as from C. elegans. This would be pivotal in understanding the dynamics of behavioral states as it has been shown that neuromodulation is essential in sustaining long-lived behavioral states and transitions between different behavioral states associated with external and internal inputs.