Accelerating Deep Neural Networks with Analog Memory Devices

Over the next few years, special-purpose hardware accelerators based on conventional digital-design techniques will optimize the GPU framework for Deep Neural Network (DNN) computations, increasing speed and reducing power for both “training” and “forward-inference.” During training, DNN weights are adjusted to improve network performance through repeated exposure to the labelled data-examples of a large dataset. During forward-inference, already trained networks are used to analyze new data-examples.

Even after the improved computational performance and efficiency that is expected from these special-purpose digital accelerators, there would still be an opportunity for even higher performance and even better energy-efficiency from neuromorphic computation based on analog memories (including both memristors and Phase-Change Memory).

In this presentation, we discuss the origin of this opportunity as well as the challenges inherent in delivering on it, with some focus on materials and devices for analog volatile and non-volatile memory. We review our group’s work towards neuromorphic chips for the hardware acceleration of training and inference of Fully-Connected DNNs [1-4]. The presentation will discuss the impact of real device characteristics – such as non-linearity, variability, asymmetry, and stochasticity – on performance, and describe how these effects determine the desired specifications for the analog resistive memories needed for this application. We present some novel solutions to finesse some of these issues in the near-term, and describe some challenges in designing and implementing the CMOS circuitry around the NVM array. The talk will end with an outlook on the prospects for analog memory-based DNN hardware accelerators.

[1] G. W. Burr et al., IEDM Tech. Digest, 29.5 (2014).
[2] G. W. Burr et al., IEDM Tech. Digest, 4.4 (2015).
[3] P. Narayanan et al., IBM J. Res. Dev., 61(4/5), 11:1-11 (2017).
[4] S. Ambrogio et al., Nature, 558(7708), 60–67 (2018).