Senior AI Evaluation Architect for Frontier Models

NVIDIA AI; Oldebroek; On-site; EUR 120,000 - 180,000; Full time

Job summary

NVIDIA seeks senior engineers to pioneer evaluation methodologies for frontier models, including LLMs, RAG, agents, and vision models. You will design decision-grade evaluation environments and build pipelines producing auditable accuracy signals that gate major model releases.

You will collaborate across teams to translate evaluation results into concrete decisions—release go/no-go, training direction, and competitive positioning—while leveraging enterprise-grade GPU clusters for scalable

Qualifications

• BS, MS, or PhD in Computer Science, Machine Learning, Statistics, or a related field.
• 6+ years of hands-on experience with LLMs, evaluating large language or multimodal AI systems.
• Strong statistical foundations: experimental design, significance testing, regression analysis.
• Proven experience building evaluation infrastructure (pipelines, CI systems, reproducible workflows).
• Clear, precise communicator who translates results to researchers, product teams, and leadership.

Responsibilities

• Design and build decision-grade evaluation environments for frontier models across reasoning, multimodal, long-context, and agentic systems.
• Research and develop novel evaluation methodologies for emerging model families where benchmarks don’t yet exist.
• Build and operate evaluation infrastructure and pipelines, including benchmark environments and CI/regression tooling.
• Partner with model research, training, and customer teams to translate evaluation signals into concrete release/go-no-go decisions.

Skills

LLMs evaluation

Evaluation infrastructure

Statistical analysis

Clear communication

HPC/Slurm experience

Education

BS/MS/PhD in CS/ML/Statistics

Tools

MLflow

Weights & Biases

Job description

NVIDIA seeks senior engineers to pioneer evaluation methodologies for frontier models, including LLMs, RAG, agents, and vision models. You will design decision-grade evaluation environments and build pipelines producing auditable accuracy signals that gate major model releases.

You will collaborate across teams to translate evaluation results into concrete decisions—release go/no-go, training direction, and competitive positioning—while leveraging enterprise-grade GPU clusters for scalable