Senior Software Engineer, Open Harness & AI Agent Eval

NVIDIA; California (MO); On-site; USD 214,000 - 357,000; Full time

Benefits offered by this job

Equity

Benefits package

Job summary

NVIDIA is seeking a Senior Software Engineer to help build core libraries, evaluation systems, and reusable harness capabilities for safer, faster AI agents. You will work hands-on across open-source agent harnesses, sandboxed execution, and model-provider infrastructure to improve agent quality, reliability, and cost across models.

You’ll diagnose failures from traces and benchmarks, contribute upstream, and promote improvements with reproducible experiments, tests, and documentation.

Qualifications

• Expert Python skills and working proficiency in Rust, Go, C++, or TypeScript.
• Experience building or extending LLM agents, coding agents, tool-use loops, model-provider integrations, or evaluation infrastructure.
• Solid understanding of asynchronous execution, subprocesses, containers, sandboxing, callbacks, retries, networking, and distributed compute.

Responsibilities

• Define and evolve the technical approach for evaluating and improving open agent harnesses across frameworks, models, and benchmark environments.
• Build and operate scalable, reproducible evaluation systems spanning CI, scheduled compute, sandboxed execution, artifact provenance, trace capture, and analysis.
• Diagnose agent failures from traces, logs, tool results, and benchmark artifacts, separating harness, evaluator, environment, inference, and model causes.
• Design controlled experiments and promotion criteria that distinguish genuine harness improvements from noise, benchmark artifacts, or model-specific gains.
• Ship focused harness and runtime improvements, including upstream open-source contributions, detectors, regression tests, and maintainable documentation.
• Partner with Relay, Hermes, model, infrastructure, and research teams to turn recurring optimization needs into reusable runtime, trace, and evaluation capabilities.

Skills

Python

Rust

Go

C++

TypeScript

Education

Bachelor’s or Master’s degree in CS/AI/Applied Math

Job description

NVIDIA is seeking a Senior Software Engineer to help build core libraries, evaluation systems, and reusable harness capabilities for safer, faster AI agents. You will work hands-on across open-source agent harnesses, sandboxed execution, and model-provider infrastructure to improve agent quality, reliability, and cost across models.

You’ll diagnose failures from traces and benchmarks, contribute upstream, and promote improvements with reproducible experiments, tests, and documentation.