Senior AI Evaluation Engineer for Frontier Models

NVIDIA; Netherlands; On-site; EUR 120,000 - 180,000; Full time

Job summary

NVIDIA is seeking senior engineers to pioneer evaluation methodologies for cutting-edge deep learning models, including LLMs and multimodal systems. You will collaborate across teams to ensure rigorous, auditable results that guide model releases and roadmap decisions.

You\'ll design evaluation environments, build pipelines, and enable rapid, reliable inference on large GPU clusters. The role offers exposure to unreleased hardware and a pivotal position in shaping AI capabilities at NVIDIA.

Qualifications

• BS, MS, or PhD in Computer Science, ML, Statistics, or related field.
• 6+ years hands-on experience with LLMs, evaluating large language models or multimodal AI systems, incl. agentic, multi-turn, or reasoning-heavy setups.
• Strong statistical foundations: experimental design, significance testing, regression analysis, distinguishing signal from noise at scale.
• Proven experience building evaluation infrastructure (pipelines, benchmark harnesses, reproducible CI) not just consuming benchmarks.
• Clear, precise communicator who translates quantitative evaluation results into decisions for researchers and leadership.

Responsibilities

• Design and build evaluation environments for NVIDIA's frontier models spanning reasoning, multimodal, long-context, and agentic systems, producing auditable accuracy signals that gate every major model release.
• Research and develop novel evaluation methodologies for emerging model families and capability domains (low-precision numerics, multi-turn agentic tasks, code generation).
• Build and operate the evaluation infrastructure and pipelines including benchmark environments, regression CI systems, and statistical analysis tooling, used by model, product, and applied research teams across NVIDIA.
• Partner with model research, training, and customer teams to translate evaluation signals into concrete decisions: release go/no-go, training iteration direction, and competitive positioning against external frontier models.

Education

BS, MS, or PhD in Computer Science, ML, Statistics, or related field

Job description

NVIDIA is seeking senior engineers to pioneer evaluation methodologies for cutting-edge deep learning models, including LLMs and multimodal systems. You will collaborate across teams to ensure rigorous, auditable results that guide model releases and roadmap decisions.

You\'ll design evaluation environments, build pipelines, and enable rapid, reliable inference on large GPU clusters. The role offers exposure to unreleased hardware and a pivotal position in shaping AI capabilities at NVIDIA.