Thesis Data Science: Investigation of Foundation Model Embeddings for suitability in the automotive context.

Context

In the automotive context, data-driven approaches are becoming increasingly important, especially for tasks such as scene understanding, anomaly detection, driving condition classification or data-based validation. Suitable representations (embeddings) that convert raw data such as camera, radar, lidar or vehicle signals into compact, informative feature spaces play a central role here. In recent years, so-called foundation models have become established, whose embedding models are pre-trained on very large and heterogeneous data sets. This data typically covers a very broad spectrum of real-world states and therefore goes far beyond the operational design domain (ODD) of a specific vehicle. The central question is therefore how well such foundation embeddings are suitable for automotive-specific tasks, what advantages (e.g. better generalization) and risks (e.g. domain gap, bias) arise and whether and how they can be used sensibly in vehicle applications.

Tasks

• Research and selection of suitable pre-trained embedding models

• Comparison of the state of the art and science regarding foundation models in automotive tasks

• Definition of suitable automotive tasks for evaluation

• Creation of an evaluation setup including suitable metrics

Requirements

• You work independently and in a structured manner, are motivated and committed.

• Python knowledge

• Previous knowledge of automotive data and practical ML use cases

• Understanding of embeddings, feature spaces and evaluation methods

• You have very good written and spoken German and English skills

• Knowledge of machine learning / statistics