Statistical Precision Annealing Method for Deep Learning

We formulate an equivalence between deep learning and state/parameter estimation problems in nonlinear dynamical systems. We also propose a general method (named Precision Annealing) inspired by this equivalence to perform learning tasks without backpropagation. Using the language of statistical physics, we then explain the novel optimization routine of the PA method and compare it with optimizations in conventional deep learning. Furthermore, we show up-to-date results of the PA method in various settings including standard deep learning tasks.