Driven by mobility and internet-of-things, edge-AI has become increasingly important. Whether it is home applications, autonomous driving or fields like sensor networks and drones, all benefit from local AI data processing close to the device or sensor.

Thus, largely reducing energy consumption avoiding high-bandwidth data transport. Autonomous driving or drones doesn’t even allow communication with bigger processing infrastructure due to real-time latency requirements.

In areas where personal data like audio, visual or even medical data is processed, it is a requirement for privacy and security to handle the data as much as possible local on the device.

However, todays AI algorithms require intensive data processing limiting the scope of edge-AI applications. Although impressive progress has been reached by technology scaling to run GPU, NPU hardware accelerators also on mobile devices, still the power consumption remains a problem to be solved. For level 5 autonomous robot taxis the overall data processing would consume appr. 30% of the battery range of today’s electric cars [1].

And even such edge-AI is still focused on “static” classification of data, e.g. images or “signatures over time”. In contrast, AI processing of time-series sensor data provides further challenges. Prediction of movement of objects in radar requires history to identify only the relevant echoes (KI-ASIC) Systems depending on complex chemical and physical properties, like batteries [2], drones or car electric drives [3], need to include them in the control & prediction loop.

Recurrent neural networks (RNN) or long-short term memory (LSTM) neural networks are applied [4] but have strong disadvantages in multiply and accumulate acceleration due to complex neuron structure of input, forget and output gates as well as limited data re-use [5]. Neuromorphic, spiking neural networks (SNN) are much more hardware friendly working in the digital domain [6], but still lacking good supervised training algorithms. Here new concepts are available like surrogate gradients [7] to allow for backpropagation training.

For a radar gesture recognition, it is shown that such training can be done with backpropagation through time.  Efficiently small networks with only 10...30 hidden leaky integrate-and-fire neurons are on-par or even able to outperform LSTM networks. The network is implemented on a digital simulation SpiNNaker 2 FPGA prototype in a real time closed loop system. Impact of quantization and buffer memory usage is investigated to optimize power and latency.

However, still it is an ongoing topic of research what applications and network architectures are best suited for SNN. Obviously, a sparse spike generation helps to reduce power and memory consumption [8]. Several approaches on spike encoding and neuron function are investigated for the gesture recognition, trading spike rate vs. accuracy [9]. For some layers, like the input layers as bitmaps, it is even beneficial to stay with e.g. CNN layers. Resulting in a hybrid CNN-SNN model still trained end-to-end.

All this learning contributes to a benchmarking of such neuromorphic approaches [10, 11]. Nevertheless, it still remains a hard problem due to large variation of use cases, platforms and training, pointing to competitions as best comparison [12].