Kreß ITIV

M. Sc. Fabian Kreß

  • Exam Date: December 19, 2024
  • Thesis Topic: "Resource-Based Inference Partitioning of Deep Neural Networks in Embedded Systems"
  • Group: Prof. Becker

  • Corrector: Prof. Dr.-Ing. Mladen Berekovic (Uni zu Lübeck)

Summary of the Dissertation

Embedded systems, such as those found in vehicles, are subject to certain requirements that they must meet. Therefore, appropriate simulation environments and tools are needed as early as the design phase to incorporate these constraints into the search for an optimal system architecture. Distributing the workload across multiple (heterogeneous) computing units is a well-known method for increasing performance and energy efficiency. 

In this context, the dissertation presents novel concepts at various design levels that examine the hardware architecture with respect to different optimization goals. The thematic focus is on system-level inference partitioning, whereby the proposed framework, CNNParted, optimizes the distribution of neural networks across available hardware resources with respect to multiple metrics. 

The case studies conducted ultimately showed that the proposed resource-based inference partitioning offers advantages over state-of-the-art methods. 

research

Exploring Design Space for AI in Embedded Systems

Applications such as object detection or classification in the division of autonomous driving are typically implemented today using artificial intelligence (AI). Unlike conventional algorithms, AI can often deliver more precise and reliable results. However, AI-based applications generally require the processing of a large number of operations. In the context of embedded platforms, it is therefore important to investigate how latency, data throughput, and power consumption can be optimized while taking into account the constraints imposed by the application.

Novel Non-Volatile Memory Technologies

Over the past few decades, novel non-volatile memory technologies (NVM), such as MRAM and ReRAM, have been developed and continuously improved. These memory devices generally consume less static power than SRAM or DRAM and require only a fraction of the area compared to an SRAM cell. Furthermore, these technologies enable efficient in-memory computing, for example, to accelerate matrix-vector multiplications. Consequently, NVM offers the opportunity to rethink established memory hierarchies and computer architectures for future systems.

Optimizing the Architecture and Toolchain of Embedded FPGAs

Embedded FPGAs (eFPGAs) increase the flexibility of the entire system because they allow for hardware reconfiguration during runtime. This makes it possible to accelerate various workloads and even update the hardware accelerators themselves. However, the initial layout of the eFPGA must be defined before tape-out. This involves not only determining the number of LUTs but also designing application-specific IPs. Integrating an eFPGA thus increases complexity during the design phase, which requires improved toolchains.

teaching

Student Projects I Have Supervised (Selection)

  • SA: “Novel Storage Technologies and Their Application in Future System Architectures”
  • BA: “Hypervisor-Based Framework for Evaluating Novel Storage Technologies”
  • SA: “Novel Memory Technologies in Future Computer Architectures”
  • MA: “A Low-Power RISC-V Processor for Tiny Machine Learning”
  • MA: “Development of an FPGA Synthesis Toolchain for the Automated Integration of Hybrid Flip-Flops”
  • SA: “Evaluation of External Memory for Running Neural Networks in IoT Devices”
  • SA: “Energy-Efficient AI Accelerators for Online Handwriting Recognition”
  • SA: “Evaluation of Optimization Strategies for Neural Networks in Digipen for Online Handwriting Recognition”
  • MA: “Hardware/Software Co-Design of an Ultra-Low-Power RISC-V Platform for Online Handwriting Recognition”

Publications


2027
Conference Papers
Data Transfer Limitations and Accuracy Trade-Offs in Neuromorphic Real-Time Systems
Weigelt, C.; Arfa, S.; Gerhards, P.; Kress, F.; Haberjahn, M.; Mayr, C.; Becker, J.
2027. Artificial Neural Networks and Machine Learning – ICANN 2026 – 35th International Conference on Artificial Neural Networks, Padua, Italy, September 14–17, 2026, Proceedings, Part I. Ed.: L. Pasa, 248–259, Springer Nature Switzerland. doi:10.1007/978-3-032-38398-3_21
2026
Conference Papers
Mitigating Event Fluctuations with Adaptive Buffering for Robust Neuromorphic Systems
Weigelt, C.; Krausse, J.; Pachideh, B.; Kress, F.; Gerhards, P.; Neher, M.; Knobloch, K.; Becker, J.
2026. Machine Learning and Principles and Practice of Knowledge Discovery in Databases : International Workshops of ECML PKDD 2025, Porto, Portugal, September 15–19, 2025, Revised Selected Papers, Part II. Ed.: I. Koprinska, 174–185, Springer Nature Switzerland. doi:10.1007/978-3-032-19099-4_13
2025
PhD Theses
Ressourcenbasierte Inferenz-Partitionierung von Deep Neural Networks in eingebetteten Systemen. PhD dissertation
Kreß, F.
2025, March 18. Karlsruher Institut für Technologie (KIT). doi:10.5445/IR/1000180033
Journal Articles
General Compilation and Mixed-Precision Partitioning: A Combined Approach for Adaptive On-Device Learning
Topko, I.; Kreß, F.; Harbaum, T.; Becker, J.
2025. IEEE Transactions on Very Large Scale Integration (VLSI) Systems, 33 (11), 2975–2983. doi:10.1109/TVLSI.2025.3611822
Conference Papers
Deep Neural Network Inference Partitioning in Embedded Hybrid Analog-Digital Systems
Kreß, F.; Hoefer, J.; Lin, Q.; Schmidt, P.; Zhu, Z.; Zhu, Y.; Harbaum, T.; Wang, Y.; Becker, J.
2025. 2025 26th International Symposium on Quality Electronic Design (ISQED), 23rd-24th April 2025, San Francisco, 1–8, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/ISQED65160.2025.11014471
2024
Journal Articles
Heterogeneous Accelerator Design for Multi-DNN Workloads via Heuristic Optimization
Balaskas, K.; Khdr, H.; Sikal, M. B.; Kreß, F.; Siozios, K.; Becker, J.; Henkel, J.
2024. IEEE Embedded Systems Letters, 16 (4), 317–320. doi:10.1109/LES.2024.3443628
Conference Papers
BayWatch: Leveraging Bayesian Neural Networks for Hardware Fault Tolerance and Monitoring
Hoefer, J.; Stammler, M.; Kreß, F.; Hotfilter, T.; Harbaum, T.; Becker, J.
2024. IEEE International Symposium on Defect and Fault Tolerance in VLSI and Technologisch Systems (DFT), Didcot, United Kingdom, 08-10 October 2024, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/DFT63277.2024.10753546
Improving Online Handwriting Trajectory Reconstruction Based on Temporal Convolutional Networks
Serdyuk, A.; Kreß, F.; Topko, I.; Harbaum, T.; Becker, J.; Hamann, T.; Kämpf, P.
2024. 2024 Sensor Data Fusion: Trends, Solutions, Applications (SDF), Bonn, Germany, 25-27 November 2024, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/SDF63218.2024.10773805
LOTTA: An FPGA-based Low-Power Temporal Convolutional Network Hardware Accelerator
Kreß, F.; Serdyuk, A.; Kobsar, D.; Hotfilter, T.; Höfer, J.; Harbaum, T.; Becker, J.
2024. 2024 IEEE 37th International System-on-Chip Conference (SOCC), Dresden, Germany, 16-19 September 2024, 126–131, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/SOCC62300.2024.10737863
Automated Deep Neural Network Inference Partitioning for Distributed Embedded Systems
Kreß, F.; El Annabi, E. M.; Hotfilter, T.; Hoefer, J.; Harbaum, T.; Becker, J.
2024. 2024 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), 1st-3rd July 2024, Knoxville, 39–44, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/ISVLSI61997.2024.00019
VHDL Crash Course: A Multimedia-Based Teaching Approach
Kreß, F.; Sidorenko, V.; Topko, I.; Unger, K.; Harbaum, T.; Becker, J.
2024. 2024 IEEE 3rd German Education Conference (GECon), Munich, Germany, 05-07 August 2024, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/GECon62014.2024.10734007
KIHT: Kaligo-Based Intelligent Handwriting Teacher
Harbaum, T.; Serdyuk, A.; Kreß, F.; Hamann, T.; Barth, J.; Kämpf, P.; Imbert, F.; Soullard, Y.; Tavenard, R.; Anquetil, E.; Delahaie, J.
2024. Proceedings - 2024 Design, Automation and Test in Europe Conference and Exhibition (DATE), 6 S., Institute of Electrical and Electronics Engineers (IEEE). doi:10.23919/DATE58400.2024.10546623
A Challenge-Based Blended Learning Approach for an Introductory Digital Circuits and Systems Course
Hoefer, J.; Gauß, M.; Adams, M.; Kreß, F.; Kempf, F.; Karle, C.; Harbaum, T.; Barth, A.; Becker, J.
2024. 2024 IEEE International Symposium on Circuits and Systems (ISCAS), Singapore, Singapore, 19-22 May 2024, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/ISCAS58744.2024.10557955
HW/SW Co-Design for Integrated AI Systems: Challenges, Use Cases and Steps Ahead
Harbaum, T.; Topko, I.; Serdyuk, A.; Fürst-Walter, I.; Kreß, F.; Becker, J.
2024. 3rd Workshop on Deep Learning for IoT (DL4IoT-2024)
Context-Aware Layer Scheduling for Seamless Neural Network Inference in Cloud-Edge Systems
Stammler, M.; Sidorenko, V.; Kreß, F.; Schmidt, P.; Becker, J.
2024. 2023 IEEE 16th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC), Singapur, 18th-21st December 2023, 97–104, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/MCSoC60832.2023.00022
Audio & Video
[VHDL Crash Course] Testbenches - How to Test your VHDL model
Kreß, F.; Sidorenko, V.; Topko, I.; Unger, K.; Schneider, M.; Becker, J.
2024
[VHDL Crash Course] Concurrent Modeling - The Register-Transfer-Level Mindset
Kreß, F.; Sidorenko, V.; Topko, I.; Unger, K.; Schneider, M.; Becker, J.
2024
[VHDL Crash Course] Sequential Modeling - Introduction to If and Case Statements
Kreß, F.; Sidorenko, V.; Topko, I.; Unger, K.; Schneider, M.; Becker, J.
2024
[VHDL Crash Course] Processes in VHDL - How to model sequential Algorithms
Kreß, F.; Sidorenko, V.; Topko, I.; Unger, K.; Schneider, M.; Becker, J.
2024
[VHDL Crash Course] Avoiding Code Duplicates - VHDL Module Parameters and Architectures
Kreß, F.; Sidorenko, V.; Topko, I.; Unger, K.; Schneider, M.; Becker, J.
2024
[VHDL Crash Course] Bit Vectors and Numbers - Basic VHDL Types
Kreß, F.; Sidorenko, V.; Topko, I.; Unger, K.; Schneider, M.; Becker, J.
2024
[VHDL Crash Course] Entity and Architecture - Introduction to the basic VHDL structure
Kreß, F.; Sidorenko, V.; Topko, I.; Unger, K.; Schneider, M.; Becker, J.
2024
[VHDL Crash Course] HDLs in general - What are HDLs used for
Kreß, F.; Sidorenko, V.; Topko, I.; Unger, K.; Schneider, M.; Becker, J.
2024
[VHDL Crash Course] How to Learn with Videos - Introduction to Self-regulated Learning
Kreß, F.; Sidorenko, V.; Topko, I.; Unger, K.; Schneider, M.; Becker, J.
2024
2023
Journal Articles
CNNParted: An open source framework for efficient Convolutional Neural Network inference partitioning in embedded systems
Kreß, F.; Sidorenko, V.; Schmidt, P.; Hoefer, J.; Hotfilter, T.; Walter, I.; Harbaum, T.; Becker, J.
2023. Computer Networks, 229, Article no: 109759. doi:10.1016/j.comnet.2023.109759
Conference Papers
ATLAS: An Approximate Time-Series LSTM Accelerator for Low-Power IoT Applications
Kreß, F.; Serdyuk, A.; Hiegle, M.; Waldmann, D.; Hotfilter, T.; Hoefer, J.; Hamann, T.; Barth, J.; Kämpf, P.; Harbaum, T.; Becker, J.
2023. 26th Euromicro Conference on Digital System Design (DSD 2023), 569–576, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/DSD60849.2023.00084
Leveraging Mixed-Precision CNN Inference for Increased Robustness and Energy Efficiency
Hotfilter, T.; Hoefer, J.; Merz, P.; Kreß, F.; Kempf, F.; Harbaum, T.; Becker, J.
2023. 2023 IEEE 36th International System-on-Chip Conference (SOCC), Santa Clara, USA, 05-08 September 2023, 1–6, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/SOCC58585.2023.10256738
Automated Replacement of State-Holding Flip-Flops to Enable Non-Volatile Checkpointing
Kreß, F.; Pfau, J.; Kempf, F.; Schmidt, P.; He, Z.; Harbaum, T.; Becker, J.
2023. 2023 IEEE Nordic Circuits and Systems Conference (NorCAS), 31st October - 1st November 2023, Aalborg, Denmark, 1–7, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/NorCAS58970.2023.10305469
A Hardware-Aware Sampling Parameter Search for Efficient Probabilistic Object Detection
Hoefer, J.; Hotfilter, T.; Kreß, F.; Qiu, C.; Harbaum, T.; Becker, J.
2023. Computer Vision Systems – 14th International Conference, ICVS 2023, Vienna, Austria, September 27–29, 2023. Ed.: H. Christensen, 299–309, Springer Nature Switzerland. doi:10.1007/978-3-031-44137-0_25
A Hardware-Centric Approach to Increase and Prune Regular Activation Sparsity in CNNs
Hotfilter, T.; Höfer, J.; Kreß, F.; Kempf, F.; Kraft, L.; Harbaum, T.; Becker, J.
2023. 2023 IEEE 5th International Conference on Artificial Intelligence Circuits and Systems (AICAS), 1–5, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/AICAS57966.2023.10168566
SiFI-AI: A Fast and Flexible RTL Fault Simulation Framework Tailored for AI Models and Accelerators
Hoefer, J.; Kempf, F.; Hotfilter, T.; Kreß, F.; Harbaum, T.; Becker, J.
2023. Proceedings of the Great Lakes Symposium on VLSI 2023, 287–292, Association for Computing Machinery (ACM). doi:10.1145/3583781.3590226
An Analytical Model of Configurable Systolic Arrays to find the Best-Fitting Accelerator for a given DNN Workload
Hotfilter, T.; Schmidt, P.; Höfer, J.; Kreß, F.; Harbaum, T.; Becker, J.
2023. DroneSE and RAPIDO: System Engineering for constrained embedded systems, 73–78, Association for Computing Machinery (ACM). doi:10.1145/3579170.3579258
Automated Search for Deep Neural Network Inference Partitioning on Embedded FPGA
Kreß, F.; Hoefer, J.; Hotfilter, T.; Walter, I.; El Annabi, E. M.; Harbaum, T.; Becker, J.
2023. Machine Learning and Principles and Practice of Knowledge Discovery in Databases. Hrsg.: I. Koprinska. Pt. 1, 557–568, Springer International Publishing. doi:10.1007/978-3-031-23618-1_37
2022
Conference Papers
Runtime Adaptive Cache Checkpointing for RISC Multi-Core Processors
Kempf, F.; Höfer, J.; Kreß, F.; Hotfilter, T.; Harbaum, T.; Becker, J.
2022. Conference Proceedings: 2022 IEEE 35th International System-on-Chip Conference (SOCC) Ed.: S. Sezer, 1–6, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/SOCC56010.2022.9908110
Data Movement Reduction for DNN Accelerators: Enabling Dynamic Quantization Through an eFPGA
Hotfilter, T.; Kreß, F.; Kempf, F.; Becker, J.; Baili, I.
2022. 2022 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), Nicosia, Cyprus, 04-06 July 2022, 371–372. doi:10.1109/ISVLSI54635.2022.00082
Hardware-aware Partitioning of Convolutional Neural Network Inference for Embedded AI Applications
Kreß, F.; Hoefer, J.; Hotfilter, T.; Walter, I.; Sidorenko, V.; Harbaum, T.; Becker, J.
2022. 18th International Conference on Distributed Computing in Sensor Systems (DCOSS), 133–140, IEEEXplore. doi:10.1109/DCOSS54816.2022.00034
Hardware-aware Workload Distribution for AI-based Online Handwriting Recognition in a Sensor Pen
Kreß, F.; Serdyuk, A.; Hotfilter, T.; Höfer, J.; Harbaum, T.; Becker, J.; Hamann, T.
2022. 2022 11th Mediterranean Conference on Embedded Computing (MECO). Ed.: IEEE, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/MECO55406.2022.9797131
Towards Reconfigurable Accelerators in HPC: Designing a Multipurpose eFPGA Tile for Heterogeneous SoCs
Hotfilter, T.; Kreß, F.; Kempf, F.; Becker, J.; Haro, J. M. De; Jiménez-González, D.; Moretó, M.; Álvarez, C.; Labarta, J.; Baili, I.
2022. 2022 Design, Automation & Test in Europe Conference & Exhibition (DATE), Antwerp, Belgium, 14-23 March 2022, 628–631, Institute of Electrical and Electronics Engineers (IEEE). doi:10.23919/DATE54114.2022.9774716
2021
Conference Papers
FLECSim-SoC: A Flexible End-to-End Co-Design Simulation Framework for System on Chips
Hotfilter, T.; Hoefer, J.; Kreß, F.; Kempf, F.; Becker, J.
2021. IEEE 34th International System-on-Chip Conference (SOCC), 14th-17th September 2021, Las Vegas, Nevada, USA, 83–88, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/SOCC52499.2021.9739212
Transparent Near-Memory Computing with a Reconfigurable Processor
Lesniak, F.; Kreß, F.; Becker, J.
2021. Applied Reconfigurable Computing. Ed.: S. Derrien, 221–231, Springer Nature Switzerland. doi:10.1007/978-3-030-79025-7_15
2018
Conference Papers
In-NoC Circuits for Low-Latency Cache Coherence in Distributed Shared-Memory Architectures
Masing, L.; Srivatsa, A.; Kreß, F.; Anantharajaiah, N.; Herkersdorf, A.; Becker, J.
2018. IEEE 12th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC), Hanoi, VN, September 12-14, 2018, 138–145, Institute of Electrical and Electronics Engineers (IEEE). doi:10.1109/MCSoC2018.2018.00033
Presentations
In-NoC circuits for low-latency cache coherence in distributed shared-memory architectures
Masing, L.; Srivatsa, A.; Kreß, F.; Anantharajaiah, N.; Herkersdorf, A.; Becker, J.
2018. 12th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC 2018), Hanoi, Vietnam, September 12–14, 2018