AI Evaluation & Safety Engineer

Syntic AI, Inc.; United States; Remote; USD 120,000 - 160,000; Full time

Job summary

Syntic AI, Inc. seeks a backend engineer to build systems that let AI agents plan tasks, call tools, retain context, and recover from errors.

This role combines agent orchestration with evaluation and safety work, aiming to make agent performance measurable, observable, interruptible, and safe in production.

You will work with TypeScript or Python, implement robust backends, and address failure modes when models use tools to ensure reliability.

Qualifications

• Experience building production-grade LLM-powered applications or agents.
• Strong backend engineering skills in TypeScript or Python.
• Experience addressing failure modes when models use tools.

Responsibilities

• Design and implement agent orchestration and tool-calling systems.
• Develop measurable evaluations to assess task success rates.
• Make agent behavior observable for evaluation of actions and outcomes.
• Build controls to interrupt agent activity safely.
• Address failure modes that arise when models use tools.

Skills

LLM-powered applications experience

Backend engineering TypeScript or Pyth

Understanding failure modes in tool-使用

Job description

Role overview

Build backend systems that let AI agents plan tasks, use tools, retain context, and recover when work goes wrong. The role combines agent orchestration with evaluation and safety work, aiming to make agent performance measurable and behavior observable, interruptible, and safe.

Responsibilities

• Design and implement agent orchestration and tool-calling systems.
• Develop measurable evaluations to assess and improve task success rates.
• Make agent behavior observable so its actions and outcomes can be assessed.
• Build controls that allow agent activity to be interrupted safely.
• Address failure modes that arise when models use tools.

Requirements

• Experience building LLM-powered applications or agents for production use.
• Strong backend engineering skills in TypeScript or Python.
• Understanding of failure modes in tool-using models.