Senior Prompt Engineer: LLM Tuning & Autorater Lead

Welo Global; United States; Remote; USD 90,000 - 130,000; Full time

Job summary

Welo Global is seeking a Prompt Engineer to manage end-to-end technical migration workflows for transitioning templates to LLM autoraters. You will leverage internal APG and APO tools to maximize model performance and ensure high-quality outputs across complex template architectures.

The role requires native Italian fluency, strong English, and a track record of 4+ years in prompt engineering, LLM tuning, and data analysis.

Qualifications

• Native fluency in Italian and fluent in English.
• Based in Italy or capable of working from Italy.
• Bachelor’s/Master’s/PhD in Computer Science, Data Science, Computational Linguistics, HCI, Cognitive Science, or related field.
• 4+ years’ experience as a Prompt Engineer.
• Experience tuning LLMs for structured outputs and complex classification tasks.

Responsibilities

• Utilize APG tools to create baseline prompts for complex template clusters.
• Run and supervise APO tool, review outputs, flag deadlocks/plateaus.
• Manually draft, test, and refine prompts for complex templates and edge cases.
• Design manual prompts to handle edge-case scenarios where tooling is lacking.
• Monitor shadowbot runs and ensure disagreements between human and LLM ratings are tracked.

Skills

Italian language

English language

Prompt engineering

LLM tuning

Data analysis

SQL

Adaptability

Communication

Education

Bachelor’s/Master’s/PhD in CS or related analytical field

Tools

Goose API

Job description

Welo Global is seeking a Prompt Engineer to manage end-to-end technical migration workflows for transitioning templates to LLM autoraters. You will leverage internal APG and APO tools to maximize model performance and ensure high-quality outputs across complex template architectures.

The role requires native Italian fluency, strong English, and a track record of 4+ years in prompt engineering, LLM tuning, and data analysis.