AI Safety Evaluations Engineer - ML Benchmarks & Tools

NVIDIA Corporation; Santa Clara (CA); On-site; USD 184,000 - 357,000; Full time

Job summary

NVIDIA Corporation in Santa Clara, CA, is seeking a Machine Learning Engineer / AI Researcher / Data Scientist to join our AI Safety and Responsibility efforts within Enterprise Risk Management. You will design and run safety benchmarks for multimodal models, translate risk scenarios into experiments, and build reproducible pipelines.

Collaborate with researchers and platform engineers to turn findings into robust training methods and safeguards.

Qualifications

• Bachelor's degree or equivalent experience.
• 6+ years in machine learning engineering or evaluation.
• Experience developing benchmarks, metrics, and comparisons for ML systems.
• Ability to translate evaluation questions into testable experiments.
• Strong Python skills and building reusable tools.

Responsibilities

• Safety Benchmarking: design and maintain safety benchmarks for large language and multimodal models.
• Evaluation Methodology: define metrics and protocols; use baselines and uncertainty estimates.
• Reproducible Evaluation: build reusable Python pipelines with versioned datasets and config.
• Automated Red-Teaming: develop tools to discover failure modes and expand evaluation coverage.
• Failure Analysis: investigate model failures and document actionable findings.
• Results and Communication: store, aggregate results, and clearly communicate findings to researchers and execs.

Skills

ML engineering

Evaluation design

Python

Data pipelines

Experiment tracking

Communication

Education

Bachelor's degree or equivalent

Job description

NVIDIA Corporation in Santa Clara, CA, is seeking a Machine Learning Engineer / AI Researcher / Data Scientist to join our AI Safety and Responsibility efforts within Enterprise Risk Management. You will design and run safety benchmarks for multimodal models, translate risk scenarios into experiments, and build reproducible pipelines.

Collaborate with researchers and platform engineers to turn findings into robust training methods and safeguards.