Staff ML Performance Engineer: Scale Training & GPU

OpenDigital Limited; Sunnyvale, Northern (CA, KY); Hybrid; USD 336,000 - 359,000; Full time

Job summary

Wayve is seeking a Staff ML Performance Engineer to join the Training Tech team in Sunnyvale, CA. You will optimize large-scale ML jobs to scale models faster, profiling workloads and implementing efficiency gains across training and inference on GPU clusters.

The role emphasizes collaboration with Research to embed training efficiency improvements, plus building observability and benchmarking tools for continuous performance discipline.

Qualifications

• 10+ years of industry experience driving performance engineering across ML systems, GPU compute infrastructure, distributed platforms or similar field.
• Experience optimizing large scale jobs on GPU compute clusters.
• Experience in working in platform teams and working with research teams.
• Experience in writing, reporting, and tracking performance benchmarks in an open and accessible way.
• Ability to write high quality, well-structured and tested Python code.
• BS or MS in Machine Learning, Computer Science, Engineering, or a related technical discipline or equivalent experience.

Responsibilities

• Profile ML workloads to identify bottlenecks and improve efficiency.
• Design and implement improvements to maximize MFU and throughput (parallelism, model compilation, mixed precision).
• Build observability tools to track MFU, throughput, latency and other metrics.
• Develop benchmarking tools to track efficiency gains or regressions.
• Collaborate with Research teams to integrate training efficiency improvements and foster performance culture.

Skills

Performance engineering

GPU compute

ML systems

Python

Benchmarks

Collaboration with Research teams

Platform teams

Distributed computing

NSight Systems

CUDA

Education

BS or MS in ML/CS/Engineering

Tools

NSight Systems

CUDA

Triton

Job description

Wayve is seeking a Staff ML Performance Engineer to join the Training Tech team in Sunnyvale, CA. You will optimize large-scale ML jobs to scale models faster, profiling workloads and implementing efficiency gains across training and inference on GPU clusters.

The role emphasizes collaboration with Research to embed training efficiency improvements, plus building observability and benchmarking tools for continuous performance discipline.