AI Safety Evaluations Engineer – Benchmarks & Reproducibility

NVIDIA Corporation; Santa Clara (CA); On-site; USD 184,000 - 288,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA Corporation is seeking an experienced Machine Learning Engineer / AI Researcher / Data Scientist to join the AI Safety and Responsibility efforts within Enterprise Risk Management. You will design safety benchmarks, define evaluation metrics, and build reproducible pipelines to study multimodal models and safeguard training methods.

You will collaborate with researchers and platform engineers to translate findings into actionable improvements and communicate risks to executives.

Qualifications

• Bachelor’s degree or equivalent experience is required.
• 6+ years in ML engineering or evaluation.
• Experience developing benchmarks, metrics, and comparisons for ML systems.
• Strong Python skills and maintainable tools and pipelines.
• Ability to communicate findings to researchers and executives.

Responsibilities

• Design and maintain safety benchmarks for large language and multimodal models.
• Define metrics and protocols to assess safety and usefulness.
• Build reusable Python pipelines with versioned data and experiment tracking.
• Develop tools for automated red-teaming and failure analysis.
• Document findings and communicate risks to researchers and leaders.

Skills

Python

Evaluation design

Data pipelines

Experiment tracking

Statistical rigor

Communication

LLM evaluation

Education

Bachelor’s degree or equivalent experience

Job description

NVIDIA Corporation is seeking an experienced Machine Learning Engineer / AI Researcher / Data Scientist to join the AI Safety and Responsibility efforts within Enterprise Risk Management. You will design safety benchmarks, define evaluation metrics, and build reproducible pipelines to study multimodal models and safeguard training methods.

You will collaborate with researchers and platform engineers to translate findings into actionable improvements and communicate risks to executives.