AI Agent Harness & Evaluation Engineer

SIVAD Consulting; San Jose (CA); On-site; USD 180,000 - 325,000; Full time

Job summary

SIVAD Consulting is hiring a Software Engineer for Agent Harness & Evaluation in San Jose, CA. You will improve AI agents, design evaluations, and translate experiments into production software.

You will work with a cross-disciplinary team of engineers, researchers, and domain experts to deliver reliable agent solutions for hardware applications, with a focus on performance and customer outcomes.

Qualifications

• Strong software engineering skills in Python and TypeScript.
• Familiarity with modern language models, agent systems, and ML workflows.
• Strong communication skills and customer judgment.
• Self-directed with ownership and product judgment.
• Experience with LLM agents, orchestration, tools, memory, tracing, ML inference, and transformer architectures.

Responsibilities

• Find failure modes of AI agents and design evaluations to improve performance, reliability, and efficiency.
• Build new agent capabilities, including tool interfaces, context management, memory, and multi-agent orchestration.
• Analyze traces and results to identify genuine improvements over noise.
• Collaborate with customers and domain experts to understand needs and ship fixes within ~3 months.
• Turn successful experiments into well-built production software.

Skills

Python

TypeScript

LLM agents

Agent orchestration

ML workflows

Communication

Customer facing

Ownership

Large-scale systems

Education

Bachelor's in CS/EE

Tools

Tracing

Experiment tooling

Job description

SIVAD Consulting is hiring a Software Engineer for Agent Harness & Evaluation in San Jose, CA. You will improve AI agents, design evaluations, and translate experiments into production software.

You will work with a cross-disciplinary team of engineers, researchers, and domain experts to deliver reliable agent solutions for hardware applications, with a focus on performance and customer outcomes.