Training-to-Learn with Memristive Devices
Emre Neftci a, Zhenming Yu a, Nathan Lereoux a
a Forschungzentrum Juelich
Proceedings of Neuromorphic Materials, Devices, Circuits and Systems (NeuMatDeCaS)
VALÈNCIA, Spain, 2023 January 23rd - 25th
Organizers: Rohit Abraham John, Irem Boybat, Jason Eshraghian and Simone Fabiano
Invited Speaker, Emre Neftci, presentation 013
DOI: https://doi.org/10.29363/nanoge.neumatdecas.2023.013
Publication date: 9th January 2023

Memristive crossbar arrays are promising non-von Neumann computing technologies to enable real-world, online
learning in neural networks. However, their deployment to real-world learning problems is hindered by their non-linearities
in conductance updates, variation during operation, fabrication mismatch and the realities of gradient descent training. In this
work, we show that, with a phenomenological model of the device and bi-level optimization, it is possible to pre-train the neural
network to be largely insensitive to such non-idealities on learning tasks. We demonstrate this effect using Model Agnostic Meta Learning (MAML) and a differentiable model of the conductance update on the Omniglot few-shot learning task. Since pre-training is a necessary procedure for any on-line learning scenario at the edge, our results may pave the way towards real-world applications of memristive devices without significant adaption overhead.

© FUNDACIO DE LA COMUNITAT VALENCIANA SCITO
We use our own and third party cookies for analysing and measuring usage of our website to improve our services. If you continue browsing, we consider accepting its use. You can check our Cookies Policy in which you will also find how to configure your web browser for the use of cookies. More info