Automated AI-based Placement of Early Exits in Deep Neural Networks

Context

Execution contexts, such as timing deadlines, energy budgets, or detected anomalies, may necessitate the need to shorten inference. While some strategies exist, like placing early exits onto existing neural networks, there is still room for improvement in terms of the accuracy of the placed early exits. Existing methods place early exits based on timing budgets and layer entropy as an estimation of expected accuracy. Dynamically learning the best placement and structure in one overarching step is expected to be promising.

Targets

The goal of this thesis is to design and implement a strategy for placing early exits on existing neural network models, adapting them to potential context changes. The task involves comparing the proposed strategy to existing methods and integrating it into a given framework.

Requirements

• Good knowledge of neural network models.
• Preferably experience using machine learning frameworks.