AI Agent Engineer – Open Source & Eval

Sail; San Francisco (CA); On-site; USD 150,000 - 230,000; Full time

Job summary

Sail is advancing AI with a focus on agentic systems and scalable evaluation. This role centers on designing custom evals for multi-turn, massively parallel agents and building harnesses to improve open models.

You’ll also help automate prompt optimization to push performance in a competitive landscape. We’re seeking someone with hands-on experience building AI agents and familiarity with open source models who can turn billions of tokens into better, more reliable answers through thoughtful

Qualifications

• Experience building AI agents.
• Familiarity with open source models.
• Ability to design evals for multi-turn, parallel agents.

Responsibilities

• Design custom evals for multi-turn, massively parallel agents.
• Build agent harnesses to improve open model performance.

Skills

AI agents

Open source models

Prompt optimization

Agent harnesses

Multi-turn evaluation

Tools

DSPy

Deepseek

Qwen

Llama

Claude Code

Job description

Sail is advancing AI with a focus on agentic systems and scalable evaluation. This role centers on designing custom evals for multi-turn, massively parallel agents and building harnesses to improve open models.

You’ll also help automate prompt optimization to push performance in a competitive landscape. We’re seeking someone with hands-on experience building AI agents and familiarity with open source models who can turn billions of tokens into better, more reliable answers through thoughtful