Evaluation and ML Systems Engineer, AI Safety and Security Engineering

NVIDIA Corporation; Santa Clara, Northern (CA, KY); Hybrid; USD 152,000 - 242,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA Corporation in Santa Clara is seeking an Evaluation/ML-Systems Engineer to own how we measure the program, building benchmarking and reproducibility systems, and defining metrics and protocols that make results meaningful and traceable.

You will design the protocols, map results to code and runs, and automate measurements so findings are reviewable. Collaboration with security researchers and platform engineers is essential.

Qualifications

• Bachelor's degree with 5+ years in ML engineering or evaluation.
• Experience designing benchmarks, metrics, and statistically sound comparisons for ML systems.
• A careful, skeptical approach to metrics, baselines, and claims.
• Solid Python engineering for shared infrastructure, including experiment tracking and data pipelines.

Responsibilities

• Design and build evaluation infrastructure for benchmarking and reproducibility.
• Define the metrics and protocols we measure against.
• Map every result to the code and runs that produced it.
• Keep findings reviewable and conclusions traceable.
• Collaborate with security researchers and platform engineers to design trustworthy experiments.
• Automate measurement workflows so teams can focus on research questions.

Skills

ML engineering

Evaluation design

Benchmarking

Metrics

Experiment tracking

Education

Bachelor's degree or equivalent experience

Tools

Python

Data pipelines

Experiment tracking

Reproducible workflows

Job description

NVIDIA is widely recognized as one of the most sought-after employers, with some of the most dedicated people in the world working for us! We believe open-weight models are foundational to American AI leadership and cybersecurity, and that trust in AI grows through participation, transparency, and broad scientific scrutiny. Our AI Safety & Security Engineering team builds and evaluates AI-powered tooling that helps find, validate, and patch software vulnerabilities. In a field full of big claims, our rule is simple: evidence first. We are looking for an Evaluation/ML-Systems Engineer to own how we measure the program. You will play a critical role in separating real capability from anecdote. You will make our numbers mean something, and keep them meaning it as the program grows. Your benchmarks will define what progress means for the program. Every conclusion the program reaches should trace back to benchmarks, runs, and code you helped make reproducible.Measurement is not a support function here; it is how the program knows what is true. You will design the metrics, choose the baselines, and write the protocols that make comparisons fair. You will build the infrastructure that keeps every result tied to the run that produced it. When a number looks surprising, your systems will make it easy to check. You will automate the boring parts of measurement so people can focus on the questions.You will partner with security researchers and platform engineers to design experiments people can trust and rerun. We review results internally before we rely on them, and your work makes that review possible. You will also help the team learn to read its own evidence honestly. Together you will build a shared habit of checking claims before repeating them.**What You'll Be Doing:*** Evaluation infrastructure: Build the benchmarking and reproducibility systems we depend on.* Metrics and protocols: Define the metrics and protocols we measure against.* Traceability: Map every result to the code and runs that produced it.* Evidence discipline: Keep findings reviewable and conclusions traceable.## **What We Need To See:*** Bachelor's degree (or equivalent experience) with 5+ years in ML engineering or evaluation.* Evaluation experience: Designing benchmarks, metrics, and statistically sound comparisons for ML systems.* Measurement rigor: A careful, skeptical approach to metrics, baselines, and claims.* Engineering skills: Solid Python engineering for shared infrastructure, including experiment tracking and data pipelines.## ## **Ways to Stand Out from the Crowd:*** Security evaluation: Exposure to evaluating security tooling or pipelines.* Agentic systems: Experience measuring agent or LLM behavior.* Community work: Contributions to public benchmarks or evaluation frameworks.With competitive salaries and a generous benefits package, NVIDIA is widely considered to be one of the technology industry's most desirable employers. We have some of the most forward-thinking and versatile people in the world working with us, and our engineering teams are growing fast in some of the most impactful fields of our generation: Confidential Computing and Data. If you're a creative engineer who enjoys autonomy and shares our passion for technology, we want to hear from you.Your base salary will be determined based on your location, experience, and the pay of employees in similar positions. The base salary range is 152,000 USD - 241,500 USD.You will also be eligible for equity and benefits.