Remote Lead Prompt Engineer - AI Strategy & Mentorship

Devconnectplatform; United States; Remote; USD 83,000 - 138,000; Part time

Job summary

Devconnectplatform seeks a Lead Prompt Engineer to guide a team through migrating templates to LLM autoraters and optimize model performance with advanced prompt techniques. You will oversee prompt generation, testing, and refinement to handle edge cases and ensure robust outputs.

The role demands 7+ years in prompt engineering, strong data analysis skills, and excellent communication. This is a remote, part-time position based in the United States, with influence on AI-driven solutions at scale.

Qualifications

• 7+ years of experience as Prompt Engineer
• Proven experience tuning LLMs for structured outputs
• Strong data analysis and SQL or similar tools
• Excellent verbal and written communication skills

Responsibilities

• Guide and manage a team through migration of templates to LLM autoraters
• Use APG/APO tools to generate and optimize prompts
• Draft and refine prompts to handle edge cases and anti-patterns
• Monitor shadowbot runs and ensure disagreements with human ratings are tracked
• Measure autorater quality against human baseline using F1, precision and recall

Skills

Prompt engineering

LLM tuning

Data analysis

Communication

Education

Master’s or Doctorate in CS/DS/HCI/Cognitive Science

Tools

Goose API

Job description

Devconnectplatform seeks a Lead Prompt Engineer to guide a team through migrating templates to LLM autoraters and optimize model performance with advanced prompt techniques. You will oversee prompt generation, testing, and refinement to handle edge cases and ensure robust outputs.

The role demands 7+ years in prompt engineering, strong data analysis skills, and excellent communication. This is a remote, part-time position based in the United States, with influence on AI-driven solutions at scale.