Agentic Systems Engineer — Orchestration & Evaluation

Jobtailor; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Jobtailor is seeking an expert to design and build agentic systems and a deployment platform. You will own end-to-end contracts for skills and evaluations, and drive scalable orchestration across agents and evaluations.

You will collaborate with Applied Science to train models and refine the harness, delivering reliable systems for production environments and enabling discovery-driven product development.

Qualifications

• Must have shipped agentic systems in production with reliability focus.
• Proven track record building tools or frameworks for engineering teams.
• Proficient in designing and managing evaluation frameworks.
• Hands-on experience building MCP implementations.
• Experience managing prompts and skills in production environments.
• Familiar with authorization frameworks and best practices.
• Comfortable with product discovery.

Responsibilities

• Define and implement skill-and-eval contract end-to-end for one vertical.
• Build an opinionated platform for agent deployment and lifecycle management.
• Develop a high-throughput orchestration engine to manage agents and evaluations at scale.
• Work at the seam between surface UX, underlying operating map, and shared platform services.
• Partner with Applied Science to train models leveraging the harness.

Skills

Agentic systems

Engineering tools

Evaluation frameworks

MCP implementations

Prompts management

Authorization frameworks

Product discovery

Tools

Agent frameworks

Frontend (React/Next.js)

Job description

Jobtailor is seeking an expert to design and build agentic systems and a deployment platform. You will own end-to-end contracts for skills and evaluations, and drive scalable orchestration across agents and evaluations.

You will collaborate with Applied Science to train models and refine the harness, delivering reliable systems for production environments and enabling discovery-driven product development.