AI Agent Evaluation & Reliability Engineer

Sivadconsulting; San Jose (CA); Vor Ort; USD 180.000 - 325.000; Vollzeit

Zusammenfassung

Sivadconsulting in San Jose, CA is seeking a Software Engineer for Agent Harness & Evaluation to advance AI agents for hardware applications. You will identify failure modes, design comprehensive evaluations, and translate experiments into production software.

You will join a growing team of software engineers, researchers, and hardware experts, collaborating across discovery, evaluation design, applied research, and product development to tackle real-world engineering challenges.

Qualifikationen

• Proficient in Python and TypeScript.
• Familiar with modern language models and agent systems.
• Experience with ML workflows and model inference.

Aufgaben

• Find failure modes of AI agents and design evaluations to improve performance, reliability, and efficiency.
• Build new agent capabilities including tool interfaces, context management, memory, and orchestration.
• Analyze traces and results to identify bottlenecks and distinguish real improvements from noise.
• Collaborate with customers and domain experts to define requirements and ship fixes end-to-end within ~3 months.
• Turn successful experiments into well-built, production-ready software.

Kenntnisse

Python

TypeScript

Language models

Agent systems

ML workflows

Communication

Customer focus

Ownership

Ausbildung

Bachelor's in CS/EE or similar

Jobbeschreibung

Sivadconsulting in San Jose, CA is seeking a Software Engineer for Agent Harness & Evaluation to advance AI agents for hardware applications. You will identify failure modes, design comprehensive evaluations, and translate experiments into production software.

You will join a growing team of software engineers, researchers, and hardware experts, collaborating across discovery, evaluation design, applied research, and product development to tackle real-world engineering challenges.