Staff AI Engineer, Agentic Systems and Evaluation

SIEL AI; United States; Remote; USD 150,000 - 210,000; Full time

Job summary

SIEL AI in the United States is seeking an experienced architect to design and own the core agent infrastructure used in client production systems. You will lead the evaluation and tracing needed to prove long-term reliability in live environments.

This hands-on role moves quickly from design to deployment, with close feedback from engineers operating the systems and a focus on production behavior as models and data evolve over time, ensuring observability, retraining triggers, and robust

Qualifications

• Strong engineering capability in agentic systems and their infrastructure.
• Experience designing orchestration, state/memory, and tool interfaces.
• Ability to build evaluations and tracing to assess reliability post-deployment.
• Focus on production behavior over time with changing models and data.
• Willingness to work closely with deploying engineers.

Responsibilities

• Build orchestration, state and memory, and tool interfaces for agentic systems.
• Design cost controls for systems that rely on models and tools.
• Develop evaluation and tracing approaches that prove system reliability.
• Detect and investigate changes in behavior after launch, including model updates.
• Support engineers deploying and debugging these systems in production.

Skills

Agentic systems engineering

Orchestration design

State/memory design

Tool interfaces

Evaluations and tracing

Production behavior monitoring

Team collaboration

Job description

Role overview

Design and own the core agent infrastructure used in client production systems, along with the evaluation and tracing needed to check that it remains reliable over time. This is a hands-on architecture role with a short path from design to deployment and close feedback from the engineers operating the systems.

Responsibilities

• Build orchestration, state and memory, and tool interfaces for agentic systems.
• Design cost controls for systems that rely on models and tools.
• Develop evaluation and tracing approaches that provide evidence of system reliability.
• Make it possible to detect and investigate changes in behavior after launch, including model updates and shifts in client data.
• Support the engineers deploying and debugging these systems in production.

Requirements

• Strong engineering capability in agentic systems and their underlying infrastructure.
• Experience designing orchestration, state or memory, and tool interfaces.
• Ability to build evaluations and tracing that help assess reliability after deployment.
• Focus on production behavior over time, including the effects of changing models and data.
• Willingness to work closely with engineers who deploy and support the systems.