Schriftspurrekonstruktion eines Sensorstiftes mit Spiking Neural Networks

Background

In the scope of this work encompasses the handwriting digitalization in real time, while writing on paper. Such a system could be helpful in everyday use and opens new possibilities for the documentation and collaboration. For this purpose, a digital pen is designed, which is equipped with inertial sensors and processing units. The implementation of the such pen poses many challenges –the precise trajectory reconstruction of the handwriting from inertial sensor data is already a complex problem. Conventional Machine Learning (ML) approaches with Artificial Neural Networks (ANN) achieving good results in this task, though the training and inference of the ANNs is too resource-intensive for small embedded systems. Recently Spiking Neural Networks (SNN) and their execution on an ultra-low-power Neuromorphic Hardware (e.g. Intel Loihi) have shown promising results, especially for the time-series data processing – similar to inertial data from a digital pen.  

Task

In the scope of this work the Spiking Neural Networks for the trajectory reconstruction of the sensor pen will be examined and implemented.

Relevant for the State-of-the-Art research are different approaches to trajectory reconstruction, especially existing ANN-approaches. Results of the research will enable the design and training of an SNN, which can solve regression problem. Furthermore, the analysis and preprocessing of the existing dataset from STABILO is a key point of the work, because the raw data could be used only after appropriate encoding, compatible with SNN. As soon as SNN-Software is implemented and evaluated, the applicability of SNNs in resource constrained embedded systems has to be analyzed.

Requirements

• Basic knowledge in ML and one of the Deep Learning frameworks (Keras + Tensorflow, Pytorch, etc)
• Experience in MEMS-sensors is a plus
• Motivation and independent working style