Physical neural networks trained with a physics-aware backpropagation algorithm

Deep neural networks approximate mathematical functions by learning the parameters of a sequence of nonlinear functions, or layers. The algorithm of choice performing this learning is the backpropagation algorithm for gradient descent. Backpropagation-based learning applied to large, deep neural networks - deep learning - has led to truly stunning progress in the last decade or so, and is the basis for countless popular techniques in modern science and industry.

We have taken a similar approach to build computations from networks of controllable physical systems - physical neural networks. The underlying idea is that the evolution of a physical system inherently performs a computation on its initial conditions, and this transformation can be adjusted by tuning degrees of freedom of that system. By combining multiple such physical transformations, and applying a backpropagation algorithm to learn their physical parameters, we can learn physical functions. Using a version of backpropagation suited to experimental physical systems, we have demonstrated physical neural networks based on nonlinear optics, analog electronics, mechanics, and coupled oscillators [Wright, Onodera et al., Nature (2022)].

In this talk, I will review this and our recent work on physical neural networks, primarily with optical systems. These include:

- The limits of optical neural networks due to quantum noise [Ma et al., arXiv:2307.15712, Wang et al., Nature Comm (2022)]

- Physical neural networks as smart sensors [Wang, Sohoni et al., Nature Photonics (2023)]

- The potential of large-scale neural networks to implement Transformer models energy-efficiently [Anderson et al., arXiv:2302.10360]

- Implementations of physical neural networks with multimode optical waves, and with systems of coupled nonlinear oscillators [Onodera, Stein et al., in prep]