About this role
Master Thesis Worker - Automated Quality Evaluation of Developer Assistant Responses
Introduction
Thesis work is an excellent way to get closer to Scania and build relationships for the future. Many of today's employees began their Scania career with their degree project.
Background
The TRATON Developer Portal provides an AI-powered Developer Assistant that helps developers find information and guidance from internal technical documentation. As its use grows, a systematic approach is needed to evaluate response quality and identify opportunities to improve both the assistant and its knowledge sources.
Objective
Develop and evaluate an automated approach for assessing Developer Assistant responses and generating actionable improvement recommendations. These recommendations should help distinguish documentation gaps from retrieval and agent-pipeline issues, supporting targeted improvements by the relevant teams.
Job description
The thesis worker will:
- Investigate methods for automated evaluation of AI-generated responses.
- Analyse conversation logs and available pipeline traces.
- Develop a prototype that identifies recurring quality issues and proposes evidence-backed actions.
- Validate the findings with experts and assess whether selected recommendations improve response quality.
Education/program/focus
Indicate education, program or focus: Master’s students in Computer Science, Software Engineering, Artificial Intelligence, Data Science, Engineering Physics or a related field. Number of students: 1 to 2 Start date for the thesis work: Jan-Feb 2027 Estimated time required: 6 months
Contact persons and supervisors
Kazi Mehedi Hasan, supervisor and fullstack developer in Traton Developer Portal team Schnell Melander Vilja, supervisor and industrial PhD student in Research team Annergren, Mariette, recruiting manager
Publication date from - to: 2026-10-26 - 2026-11-20
Requisition ID:
34435
Number of Openings:
1.0
Part-time / Full-time:
Full-time
Permanent / Temporary:
Temporary
Country / Region:
SE
Location(s):
Södertälje, SE, 151 38
Required Travel:
0-25%
Workplace:
On-site