Training photonic and electronic stochastic physical neural networks

The computational demands of deep learning motivate the investigation of alternative approaches to computation. One alternative is physical neural networks (PNNs) in which computation and learning are performed directly via a physical process. Stochastic PNNs are created when the underlying neurons are realized by the stochastic behaviour of an activation switch. We propose novel photonic and electronic activation switches, with the photonic realization created by sideband driving of an optomechanical system and the electronic realization created by single-electron tunnelling through a quantum dot. In the optomechanical case, the occupation of the optical and/or mechanical mode is the basis for the activation switch. Training of stochastic PNNs is simulated with these activation switches, as well as with single photon detection activation switches. Several training approaches for MNIST handwritten digit classification have been investigated, for example, using various number of trials in each layer and using either true probability or empirical outputs in the backward pass. High test accuracy has been demonstrated in the presence of a high degree of noise and model uncertainty. The results demonstrate the potential of embracing stochastic physical systems for deep learning.