AI Agent Safety & Evaluation Engineer

Syntic AI, Inc.; United States; Remote; USD 120,000 - 160,000; Full time

Job summary

Syntic AI, Inc. seeks a backend engineer to build systems that let AI agents plan tasks, call tools, retain context, and recover from errors.

This role combines agent orchestration with evaluation and safety work, aiming to make agent performance measurable, observable, interruptible, and safe in production.

You will work with TypeScript or Python, implement robust backends, and address failure modes when models use tools to ensure reliability.

Qualifications

• Experience building production-grade LLM-powered applications or agents.
• Strong backend engineering skills in TypeScript or Python.
• Experience addressing failure modes when models use tools.

Responsibilities

• Design and implement agent orchestration and tool-calling systems.
• Develop measurable evaluations to assess task success rates.
• Make agent behavior observable for evaluation of actions and outcomes.
• Build controls to interrupt agent activity safely.
• Address failure modes that arise when models use tools.

Skills

LLM-powered applications experience

Backend engineering TypeScript or Pyth

Understanding failure modes in tool-使用

Job description

Syntic AI, Inc. seeks a backend engineer to build systems that let AI agents plan tasks, call tools, retain context, and recover from errors.

This role combines agent orchestration with evaluation and safety work, aiming to make agent performance measurable, observable, interruptible, and safe in production.

You will work with TypeScript or Python, implement robust backends, and address failure modes when models use tools to ensure reliability.