Making neuromorphic the main stream of AI

Timoleon Moraitis^a, Qinghai Guo^a, Hector Garcia Rodriguez^a^,^b, Franz Scherr^a, Adrien Journé^a, Pontus Stenetorp^b, Yansong Chua^a, Dmitry Toichkin^c

^aHuawei Technologies, Zürich, Switzerland, Zürich, Switzerland

^bUniversity College London, Roberts building, Torrington Place, London, United Kingdom

^cNone, United Kingdom

Proceedings of Neuromorphic Materials, Devices, Circuits and Systems (NeuMatDeCaS)

VALÈNCIA, Spain, 2023 January 23rd - 25th

Organizers: Rohit Abraham John, Irem Boybat, Jason Eshraghian and Simone Fabiano

Contributed talk, Timoleon Moraitis, presentation 049
DOI: https://doi.org/10.29363/nanoge.neumatdecas.2023.049
Publication date: 9th January 2023

Neuromorphic computing is widely regarded as a constraint that increases AI efficiency but trades off proficiency, such as accuracy. However, this separates neuromorphic use cases from mainstream AI, into niches where proficiency is not crucial. To the contrary, our recent work [1-5] shows that neuromorphic mechanisms do not constrain but rather they expand and improve mainstream AI in conventional measures of proficiency, while also improving efficiency. Specifically, we show that neuromorphic algorithms can outperform the state of the art of the broader, conventional machine learning field, beyond neuromorphic niches, in classification accuracy, robustness, inference speed, learning speed, and task reward. We show these in tasks such as keyword spotting, ImageNet classification, playing Atari games, controlling robots, many of which are usually out of reach for neuromorphic models. We test these in settings of online adaptation, supervised, unsupervised, self-supervised or reinforcement learning, meta-learning, and non-backprop-based deep learning. This is achieved by exploiting also other biological mechanisms than spikes. The biological mechanisms that we employ include spikes, Hebbian plasticity, short-term plasticity, and efference copies. The algorithms remain suitable for efficient neuromorphic hardware, and include a new method for decreasing the power consumption of weighting operations in analog synapses. Based on these results, we propose that exploiting the breadth of neuromorphic mechanisms, besides spikes, in suitable applications, is a path towards making neuromorphic the mainstream type of AI computing.

References:

[1] Jeffares, A., Guo, Q., Stenetorp, P., & Moraitis, T. (2022). Spike-inspired Rank Coding for Fast and Accurate Recurrent Neural Networks. International Conference on Learning Representations (ICLR 2022)

[2] Rodriguez, H.G., Guo, Q., & Moraitis, T. (2022). Short-Term Plasticity Neurons Learning to Learn and Forget. Proceedings of the 39th International Conference on Machine Learning (ICML 2022), 162:18704-18722

[3] Moraitis, T., Toichkin, D., Journé, A., Chua, Y., & Guo, Q. (2022). SoftHebb: Bayesian inference in unsupervised Hebbian soft winner-take-all networks. Neuromorphic Computing and Engineering.

[4] Journé, A., Rodriguez, H. G., Guo, Q., & Moraitis, T. (2023). Hebbian deep learning without feedback. International Conference on Learning Representations (ICLR 2023)

[5] Scherr, F., Guo, Q., & Moraitis, T. (2022). Self-supervised learning through efference copies. Advances in Neural Information Processing Systems (NeurIPS 2022)

nanoGe is a prestigious brand of successful science conferences that are developed along the year in different areas of the world since 2009. Our worldwide conferences cover cutting-edge materials topics like perovskite solar cells, photovoltaics, optoelectronics, solar fuel conversion, surface science, catalysis and two-dimensional materials, among many others.

MATSUS

Previously nanoGe Spring Meeting (NSM) and nanoGe Fall Meeting (NFM), MATSUS is a multiple symposia conference focused on a broad set of topics of advanced materials preparation, their fundamental properties, and their applications, in fields such as renewable energy, photovoltaics, lighting, semiconductor quantum dots, 2-D materials synthesis, charge carriers dynamics, microscopy and spectroscopy semiconductors fundamentals, etc.

International Conference on Hybrid and Organic Photovoltaics

International Conference on Hybrid and Organic Photovoltaics (HOPV) is celebrated yearly in May. The main topics are the development, function and modeling of materials and devices for hybrid and organic solar cells. The field is now dominated by perovskite solar cells but also other hybrid technologies, as organic solar cells, quantum dot solar cells, and dye-sensitized solar cells and their integration into devices for photoelectrochemical solar fuel production.

Asia-Pacific International Conference on Perovskite, Organic Photovoltaics and Optoelectronics

The main topics of the Asia-Pacific International Conference on Perovskite, Organic Photovoltaics and Optoelectronics (IPEROP) are discussed every year in Asia-Pacific for gathering the recent advances in the fields of material preparation, modeling and fabrication of perovskite and hybrid and organic materials. Photovoltaic devices are analyzed from fundamental physics and materials properties to a broad set of applications. The conference also covers the developments of perovskite optoelectronics, including light-emitting diodes, lasers, optical devices, nanophotonics, nonlinear optical properties, colloidal nanostructures, photophysics and light-matter coupling.

International Conference on Perovskite Thin Film Photovoltaics Perovskite Photonics and Optoelectronics

The International Conference on Perovskite Thin Film Photovoltaics Perovskite Photonics and Optoelectronics (NIPHO) is the best place to hear the latest developments in perovskite solar cells as well as on recent advances in the fields of perovskite light-emitting diodes, lasers, optical devices, nanophotonics, nonlinear optical properties, colloidal nanostructures, photophysics and light-matter coupling.