Senior AI Evaluation Architect for Frontier Models

NVIDIA Corporation; Poland; Remote; PLN 375,000 - 650,000; Full time

Job summary

NVIDIA Corporation is seeking senior engineers to design evaluation environments for cutting-edge AI models, including LLMs, multimodal systems, and agents. You will build infrastructure, pursue novel evaluation methods, and collaborate with research and product teams to influence model releases.

The role emphasizes strong statistical analysis, scalable benchmarking, and clear communication of results to researchers and leadership, with engagement across a fast-growing AI ecosystem.

Qualifications

• 6+ years of hands-on experience with LLMs and evaluating large language or multimodal AI systems.
• Experience building evaluation infrastructure and CI systems.
• Strong statistical design, significance testing, and benchmarking analysis.

Responsibilities

• Design and build evaluation environments for frontier models and responsible signaling.
• Develop novel evaluation methodologies for emerging model families and capabilities.
• Build and operate evaluation pipelines, benchmarks, and reproducible CI for cross-functional teams.

Skills

LLMs experience

Evaluation design

Statistical methods

Clear communicator

Education

BS/MS/PhD in CS/ML/Statistics

Tools

MLflow

W&B

Job description

NVIDIA Corporation is seeking senior engineers to design evaluation environments for cutting-edge AI models, including LLMs, multimodal systems, and agents. You will build infrastructure, pursue novel evaluation methods, and collaborate with research and product teams to influence model releases.

The role emphasizes strong statistical analysis, scalable benchmarking, and clear communication of results to researchers and leadership, with engagement across a fast-growing AI ecosystem.