Senior Prompt Engineer – LLM Evaluation & APO (Germany)

Welo Global; United States; Remote; USD 78,000 - 112,000; Full time

Job summary

Welo Global is seeking a Prompt Engineer based in Germany to lead the end-to-end migration of templates to LLM autoraters. You will use client internal tools and advanced prompting techniques to maximize model performance in a vendor setting.

The role requires 4+ years in prompt engineering, native German and fluent English, and a relevant CS/DS degree. You will craft, test, and refine prompts, monitor APO and shadowbot outputs, and document launches for approval.

Qualifications

• Must have 4+ years as a Prompt Engineer.
• Experience tuning LLMs for structured outputs and classification.
• Familiarity with chain-of-thought and few-shot learning.
• Strong data analysis skills using SQL or similar tools.

Responsibilities

• Operate end-to-end migration workflow for templates to LLM autoraters.
• Use client tools to maximize model performance via prompt engineering.
• Create, test, and refine prompts for complex template clusters.
• Monitor and manage Automated Prompt Optimization outputs.
• Analyze model errors and track metrics like F1, precision, recall.
• Prepare Launch Certification Documentation for deployment.

Skills

German (native)

English (fluent)

Prompt engineering

LLM tuning

SQL analytics

Communication

Education

Bachelor’s/Master’s/PhD in CS/DS/Computational Linguistics/HCI/Cognitive Science

Tools

Goose API

APO systems

SQL tooling

Job description

Welo Global is seeking a Prompt Engineer based in Germany to lead the end-to-end migration of templates to LLM autoraters. You will use client internal tools and advanced prompting techniques to maximize model performance in a vendor setting.

The role requires 4+ years in prompt engineering, native German and fluent English, and a relevant CS/DS degree. You will craft, test, and refine prompts, monitor APO and shadowbot outputs, and document launches for approval.