Integration of physics-derived memristor models with machine learning frameworks

Zhenming Yu^a^,^b, Stephan Menzel^a, John Paul Strachan^a^,^b, Emre Neftci^a^,^b

^aForschungszentrum Jülich GmbH, Wilhelm-Johnen-Straße, Jülich, Germany

^bRWTH Aachen University, Faculty of Electrical Engineering and Information Technology, Templergraben, 55, Aachen, Germany

Proceedings of Neuromorphic Materials, Devices, Circuits and Systems (NeuMatDeCaS)

VALÈNCIA, Spain, 2023 January 23rd - 25th

Organizers: Rohit Abraham John, Irem Boybat, Jason Eshraghian and Simone Fabiano

Contributed talk, Zhenming Yu, presentation 011
DOI: https://doi.org/10.29363/nanoge.neumatdecas.2023.011
Publication date: 9th January 2023

Simulation frameworks such MemTorch [1] [2], DNN+NeuroSim [3] [4], and aihwkit [5] are commonly used to facilitate the end-to-end co-design of memristive machine learning (ML) accelerators. These simulators can take device nonidealities into account and are integrated with modern ML frameworks. However, memristors in these simulators are modeled with either lookup tables or simple analytic models with basic nonlinearities. These simple models are unable to capture certain performance critical aspects of device nonidealities. For example, they ignore the physical cause of switching, which induces errors in switching timings and thus incorrect estimations of conductance states. This work aims at bringing physical dynamics into consideration to model nonidealities while being compatible with GPU accelerators. We focus on Valence Change Memory (VCM) cells, where the switching nonlinearity and SET/RESET asymmetry relate tightly with the thermal resistance, ion mobility, Schottky barrier height, parasitic resistance, and other effects [6]. The resulting dynamics require solving an ODE that captures changes in oxygen vacancies. We modified a physics-derived SPICE-level VCM model [7] [8], integrated it with the aihwkit [5] simulator and tested the performance with the MNIST dataset. Results show that noise that disrupts the SET/RESET matching affects network performance the most. This work serves as a tool for evaluating how physical dynamics in memristive devices affect neural network accuracy and can be used to guide the development of future integrated devices.

References:

[1] C. Lammie and M. R. Azghadi, “Memtorch: A simulation framework for deep memristive cross-bar architectures,” in 2020 IEEE international symposium on circuits and systems (ISCAS). IEEE, 2020, pp. 1–5.

[2] C. Lammie, W. Xiang, B. Linares-Barranco, and M. R. Azghadi, “Memtorch: An open-source simulation framework for memristive deep learning systems,” Neurocomputing, vol. 485, pp. 124–133, 2022.

[3] X. Peng, S. Huang, H. Jiang, A. Lu, and S. Yu, “Dnn+ neurosim v2. 0: An end-to-end benchmarking framework for compute-in-memory accelerators for on-chip training,” IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 40, no. 11, pp. 2306–2319, 2020.

[4] S. Yu, H. Jiang, S. Huang, X. Peng, and A. Lu, “Compute-in-memory chips for deep learning: Recent trends and prospects,” IEEE Circuits and Systems Magazine, vol. 21, no. 3, pp. 31–56, 2021.

[5] M. J. Rasch, D. Moreda, T. Gokmen, M. Le Gallo, F. Carta, C. Goldberg, K. El Maghraoui, A. Sebastian, and V. Narayanan, “A flexible and fast pytorch toolkit for simulating training and inference on analog crossbar arrays,” in 2021 IEEE 3rd international conference on artificial intelligence circuits and systems (AICAS). IEEE, 2021, pp. 1–4.

[6] F. C¨uppers, S. Menzel, C. Bengel, A. Hardtdegen, M. Von Witzleben, U. B¨ottger, R. Waser, and S. Hoffmann-Eifert, “Exploiting the switching dynamics of hfo2-based reram devices for reliable analog memristive behavior,” APL materials, vol. 7, no. 9, p. 091105, 2019.

[7] C. Bengel, A. Siemon, F. C¨uppers, S. Hoffmann-Eifert, A. Hardtdegen, M. von Witzleben, L. Hellmich, R. Waser, and S. Menzel, “Variability-aware modeling of filamentary oxide-based bipolar resistive switching cells using spice level compact models,” IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 67, no. 12, pp. 4618–4630, 2020.

[8] V. Ntinas, A. Ascoli, I. Messaris, Y. Wang, V. Rana, S. Menzel, and R. Tetzlaff, “Towards simplified physics-based memristor modeling of valence change mechanism devices,” IEEE Transactions on Circuits and Systems II: Express Briefs, 2022.

Acknowledgements:

This work was sponsored by the Federal Ministry of Education, Germany (project NEUROTEC-II grant no. 16ME0398K and 16ME0399). We thank Vasileios Ntinas for his help with the simplified model [8], and Malte J. Rasch for his generous support on aihwkit [5].

nanoGe is a prestigious brand of successful science conferences that are developed along the year in different areas of the world since 2009. Our worldwide conferences cover cutting-edge materials topics like perovskite solar cells, photovoltaics, optoelectronics, solar fuel conversion, surface science, catalysis and two-dimensional materials, among many others.

MATSUS

Previously nanoGe Spring Meeting (NSM) and nanoGe Fall Meeting (NFM), MATSUS is a multiple symposia conference focused on a broad set of topics of advanced materials preparation, their fundamental properties, and their applications, in fields such as renewable energy, photovoltaics, lighting, semiconductor quantum dots, 2-D materials synthesis, charge carriers dynamics, microscopy and spectroscopy semiconductors fundamentals, etc.

International Conference on Hybrid and Organic Photovoltaics

International Conference on Hybrid and Organic Photovoltaics (HOPV) is celebrated yearly in May. The main topics are the development, function and modeling of materials and devices for hybrid and organic solar cells. The field is now dominated by perovskite solar cells but also other hybrid technologies, as organic solar cells, quantum dot solar cells, and dye-sensitized solar cells and their integration into devices for photoelectrochemical solar fuel production.

Asia-Pacific International Conference on Perovskite, Organic Photovoltaics and Optoelectronics

The main topics of the Asia-Pacific International Conference on Perovskite, Organic Photovoltaics and Optoelectronics (IPEROP) are discussed every year in Asia-Pacific for gathering the recent advances in the fields of material preparation, modeling and fabrication of perovskite and hybrid and organic materials. Photovoltaic devices are analyzed from fundamental physics and materials properties to a broad set of applications. The conference also covers the developments of perovskite optoelectronics, including light-emitting diodes, lasers, optical devices, nanophotonics, nonlinear optical properties, colloidal nanostructures, photophysics and light-matter coupling.

International Conference on Perovskite Thin Film Photovoltaics Perovskite Photonics and Optoelectronics

The International Conference on Perovskite Thin Film Photovoltaics Perovskite Photonics and Optoelectronics (NIPHO) is the best place to hear the latest developments in perovskite solar cells as well as on recent advances in the fields of perovskite light-emitting diodes, lasers, optical devices, nanophotonics, nonlinear optical properties, colloidal nanostructures, photophysics and light-matter coupling.