Decoding human behavior from complex neural interactions

The brain is a complex system with intricate neural dynamics generated by interactions. How these interactions ultimately control an animal’s behavior, however, remains unresolved. This mapping is particularly mysterious for behaviors involving the coordination of multiple body parts over time scales longer than tens of milliseconds. Here we use electrocorticography (ECoG) data recorded from multiple patients to create a 2D representation of the stereotyped states of neural activity, based on embedding theorems in nonlinear state-space reconstruction. This method captures structured interactions underlying brain-wide dynamics that may be missed by conventional correlation-based analysis. Videos are used to decode human activities in an unsupervised manner. We discover patterns in the neural map that can be matched to behaviorally salient categories such as movement, speech, and rest. The transition of states is projected back to the neural map itself. Optimal predictive representations that best describe the hierarchical relationship between states over the multiple time scales are found in these transitions. These measurements are in agreement with theories from ethology and cognitive psychology, pointing towards general principles of the neural control of behavior.