Master thesis - End-to-End Autonomous Driving with Vision Foundation Models and Generative Planning

30 credits - End-to-End Autonomous Driving with Vision Foundation Models and Generative Planning

Background

Autonomous driving is undergoing a paradigm shift. While traditional autonomous driving systems rely on separate modules for perception, prediction, planning, and control, recent advances in deep learning have demonstrated the potential of end-to-end models that directly predict driving behavior from sensor observations.

At the same time, vision foundation models, such as V-JEPA, have shown impressive capabilities in learning rich visual representations from large-scale image and video data without manual annotations. In parallel, generative planning approaches based on diffusion and flow matching have emerged as powerful methods for producing accurate and diverse driving trajectories.

These developments present exciting opportunities for designing the next generation of autonomous driving systems, but many research questions remain open.

Objective

The objective of this thesis is to investigate how modern vision foundation models and generative planning techniques can be leveraged to improve end-to-end autonomous driving. The exact research questions will be formulated together with the student based on current literature, the student’s interests, and the project direction.

Possible research directions include:

• Leveraging pre-trained vision foundation models (e.g., V-JEPA) for end-to-end autonomous driving.

• Investigating diffusion- or flow matching-based approaches for trajectory generation.

• Developing novel architectures for combining visual representations with generative planners.

• Studying robustness, generalization, efficiency, or data requirements of modern end-to-end driving models.

• Evaluating the proposed methods on large-scale autonomous driving datasets.

The Project Offers

• The opportunity to work with state-of-the-art deep learning methods for autonomous driving.

• Access to large-scale driving datasets and modern GPU computing infrastructure.

• Close collaboration with researchers working on next-generation AI for autonomous vehicles.

• A chance to contribute to an active research area with potential for scientific publication.

Education/program/focus

We are looking for one or two motivated Master’s students in Computer Science, Robotics, Engineering Physics, Electrical Engineering, Applied Mathematics, or a related field.

Experience with one or more of the following is beneficial:

• Deep learning and machine learning

• Computer vision

• Generative AI

• PyTorch or similar frameworks

• Autonomous driving or robotics

Number of students: 1

Start date for the thesis work: [To be agreed]

Estimated time required: 20 weeks, full time (30 credits)

Contact persons and supervisors

Truls Nyberg, truls.nyberg@scania.com; Samir Khays, samir.khays@scania.com; Giulia D’Ascenzi, giulia.dascenzi@scania.com

Hiring Manager: Magnus Granström, magnus.granstrom@scania.com

Application

Your application must include a CV, personal letter, and transcript of grades.

A background check might be conducted for this position. We are conducting interviews continuously and may close the recruitment earlier than the date specified.