Maximum Entropy Inference of Human Decision Policies in Spatial Exploration Tasks

Advancements in high speed and high resolution motion capture as well as computer vision and deep learning engendered the study of computational ethology. A field which examines behavior through the lens of computational models. Our work aims to investigate parameters that drive human navigation in a museum setting using reinforcement learning (RL) techniques. We used recordings of patrons perusing a museum gallery. Using ideas from Maximum Entropy Inverse Reinforcement Learning (MaxEnt IRL) we regard them as agents who behave with the goal of seeking optimal reward. Therefore we seek to resolve the reward structure of the state space, and quantify their policy π(a|s), a distribution of actions, a, in a given state, s. We overcome the challenge of capturing natural human behavior through simple models such as these through novel feature engineering techniques. We use the Information Bottleneck framework, to make quantitative statements about the source of complex human behavior. We do this by analyzing policies in terms of information theoretic quantities such as information gain and decision complexity. As a result we present an interpretable frame-work for analyzing and recreating human behavioral traces. We plan to generalize this model for other spaces and incorporate complex visual features.