Prompt Engineer: LLM Migration & Optimization

Welo Data; Deutschland; Vor Ort; EUR 68.393 - 102.590; Teilzeit

Zusammenfassung

Welo Data is seeking a technical Prompt Engineer to lead the migration of workflows into high-performance LLM autoraters. You will utilize advanced APG/APO tools to optimize the processes, ensuring systems meet human accuracy standards. Ideal candidates should have 2+ years of experience, a strong academic background in relevant fields, and proficiency in SQL and internal tools like the Goose API. This is a remote, part-time freelance opportunity with flexible hours.

Qualifikationen

• 2+ years of experience as a Prompt Engineer.
• Comfortable tuning LLMs for structured outputs and complex classification tasks.
• Fast learner capable of mastering proprietary internal tools.

Aufgaben

• Lead the end-to-end migration of template workflows into LLM autoraters.
• Run and supervise Automated Prompt Optimization (APO) tools.
• Continuously measure quality against gold data and calculate performance metrics.

Kenntnisse

Linguistic & AI Mastery

Technical Agility

Data Fluency

Ausbildung

BS, MS, or PhD in Computer Science, Data Science, Computational Linguistics

Tools

SQL

Goose API

Jobbeschreibung

United States • Remote • Part-Time

Are you an expert at navigating the complex logic of Large Language Models? Welo Data is seeking a technical Prompt Engineer to lead the end‑to‑end migration of template workflows into high‑performance LLM autoraters.

This isn’t just about writing prompts—it’s about engineering a technical bridge. You will use advanced APG/APO tools and manual refinement to ensure our automated systems meet (and exceed) human accuracy baselines.

The Mission: Architecting the Future of Rating

• Technical Migration: Take ownership of the workflow for transitioning templates to LLM autoraters.
• Optimization Leadership: Run and supervise Automated Prompt Optimization (APO) tools, identifying where logic plateaus and providing the manual "spark" to overcome deadlocks.
• Metrics‑Driven Accuracy: Continuously measure quality against gold data, calculating critical performance metrics like precision, recall, and $F_1$ scores.
• Edge‑Case Engineering: Solve complex scenarios by designing manual prompts that handle anti‑patterns and broken logic in legacy architectures.

Project Details

• Schedule: Part‑Time (Flexible hours within project milestones)
• Location: 100% Remote (Must be based in the United States)
• Employment Type: Freelance / Independent Contractor

Who We Are Looking For

• Linguistic & AI Mastery: 2+ years of experience as a Prompt Engineer. You must be comfortable tuning LLMs for structured outputs and complex classification tasks.
• Academic Background: BS, MS, or PhD in Computer Science, Data Science, Computational Linguistics, or a related analytical field.
• Technical Agility: Fast learner capable of mastering proprietary internal tools and interfaces (like the Goose API) with minimal supervision.
• Data Fluency: Strong ability to identify error patterns and use SQL or data analytics tools to analyze model performance.

Preferred Skills

• Familiarity with shadowbot disagreement tracking between humans and LLMs.
• Hands‑on experience with Chain-of-Thought (CoT) and few‑shot learning.
• Proven ability to draft high-level Launch Certification Documentation.