Labeling & Evaluation Agent (German)

David Kennedy Recruitment is working with a leading technology company specialising in artificial intelligence that is seeking to onboard a Labeling & Evaluation Agent to support the development and continuous improvement of its AI models.

The role focuses on evaluating AI-generated outputs, including chatbot responses, and providing high-quality annotations and feedback. You will assess response quality, factual accuracy, relevance, safety and policy compliance, helping improve the performance and reliability of large language models (LLMs).

Position: Labeling & Evaluation Agent (German)

Location: Lisbon, Portugal

Work model: On-site

Employment type: Full-time

DUTIES AND RESPONSIBILITIES:

• Evaluate AI-generated responses using detailed quality assessment and annotation guidelines

• Label and annotate language model outputs accurately and consistently

• Assess generated content for quality, relevance, factual accuracy, safety and policy compliance

• Apply complex annotation guidelines across different scenarios and content types

• Identify inconsistencies, edge cases and potential issues, escalating them when necessary

• Analyse structured text, language data and AI-generated content

• Maintain strict confidentiality and comply with project standards and data security requirements

• Provide accurate evaluations, annotations and actionable feedback to support ongoing AI model improvement

REQUIREMENTS

• Native-level proficiency in German

• English proficiency at B2 level or above

• Strong understanding of Generative AI, large language models (LLMs) and AI annotation or evaluation platforms

• Excellent analytical and critical thinking skills, with strong attention to detail

• Ability to make objective, evidence-based decisions

• Ability to interpret and consistently apply complex written guidelines and policies

• Strong reading comprehension and written language skills

• Ability to evaluate structured text, datasets and AI-generated content

• Ability to meet productivity, accuracy and quality targets

• Professionalism, integrity and discretion when handling confidential information

BENEFITS

• Modern office in city center with open spaces, easy to access with public transportation

• Comprehensive initial training and guidance to make sure you're well prepared and comfortable at a new workplace

• Full-time employment in Lisbon

• 24/7 rotational working schedule

• 2 rotational days off

• Opportunity to work in the rapidly developing field of Generative AI and Large Language Models

• International and multicultural working environment

• Support from colleagues with coaching and mentoring opportunities

• Health and Wellbeing initiatives

• Monthly meal allowance

• And much more!!!