Senior Software Engineer, Open Agent Harness & Evaluation

NVIDIA Corporation; California; Hybrid; USD 184,000 - 357,000; Full time

Job summary

NVIDIA Corporation is seeking a Senior Software Engineer to help build core libraries, evaluation systems, and reusable harness capabilities for safer, faster AI agents. You will work hands-on across open‑source agent harnesses, sandboxed execution, and model‑provider infrastructure to improve agent evaluation quality, reliability, and latency.

As part of a collaborative engineering team, you will diagnose failures from traces and logs, design experiments, and ship upstream improvements while

Qualifications

• Bachelor's or Master's degree in Computer Science, AI, Applied Math, or related field.
• 8+ years hands-on software‑engineering experience with production systems ownership.
• Expert Python skills and working proficiency in Rust, Go, C++, or TypeScript.
• Experience building or extending LLM agents, tool‑use loops, model‑provider integrations, or evaluation infrastructure.
• Solid understanding of asynchronous execution, containers, sandboxing, networking, and distributed compute.
• Experience crafting reproducible experiments, benchmark methodology, and reliability analysis under nondeterministic conditions.
• Record of shipping and maintaining open‑source or developer‑facing software with testing, API development, docs, and code reviews.

Responsibilities

• Define and evolve the technical approach for evaluating and improving open agent harnesses across frameworks, models, and benchmark environments.
• Build and operate scalable, reproducible evaluation systems spanning CI, scheduled compute, sandboxed execution, artifact provenance, trace capture, and analysis.
• Diagnose agent failures from traces, logs, tool results, and benchmark artifacts, identifying harness, evaluator, environment, inference, and model causes.
• Design controlled experiments and promotion criteria to distinguish genuine harness improvements from noise or model-specific gains.
• Ship focused harness and runtime improvements, including upstream open‑source contributions, detectors, regression tests, and maintainable docs.
• Partner with Relay, Hermes, model, infrastructure, and research teams to turn recurring optimization needs into reusable runtime, trace, and evaluation capabilities.

Skills

Python

Rust

Go

C++

TypeScript

Education

Bachelor’s or Master’s in CS/AI or related field

Tools

SWE-bench

Terminal-Bench

AgentBench

OpenTelemetry

Job description

NVIDIA Corporation is seeking a Senior Software Engineer to help build core libraries, evaluation systems, and reusable harness capabilities for safer, faster AI agents. You will work hands-on across open‑source agent harnesses, sandboxed execution, and model‑provider infrastructure to improve agent evaluation quality, reliability, and latency.

As part of a collaborative engineering team, you will diagnose failures from traces and logs, design experiments, and ship upstream improvements while