DOI: https://doi.org/10.29363/nanoge.neuronics.2024.027
Publication date: 18th December 2023
Memristive crossbar arrays show promise as non-von Neumann computing technologies, bringing sophisticated neural network processing to the edge and facilitating real-world online learning. However, their deployment for real-world learning problems faces challenges such as non-linearities in conductance updates, variations during operation, fabrication mismatch, conductance drift, and the realities of gradient descent training.
This talk will present methods to pre-train neural networks to be largely insensitive to these non-idealities during learning tasks. These methods rely on a phenomenological model of the device, obtainable experimentally, and bi-level optimization. We showcase this effect through meta-learning and a differentiable model of conductance updates on few-shot learning tasks. Since pre-training is a necessary procedure for any online learning scenario at the edge, our results may pave the way for real-world applications of memristive devices without significant adaptation overhead.
Furthermore, by considering the programming of memristive devices as a learning problem in its own right, we demonstrate that the developed methods can accelerate existing write-verify techniques.
This work was sponsored by the Federal Ministry of Education, Germany (project NEUROTEC-II grant no. 16ME0398K and 16ME0399) and Neurosys as part of the initiative "Cluster4Future" funded by the Federal Ministery of Education and Research BMBF (03ZU1106CB).