Staff Software Engineer, TPU Performance & ML Infrastructure

Google; New York (NY); On-site; USD 207,000 - 300,000; Full time

Job summary

Google is seeking a Staff Software Engineer, TPU, Performance to advance ML performance on TPU systems and contribute to Gemini and OSS ML models. You will work on optimizing model architectures, tuning compilers and runtime for large-scale ML workloads, and collaborating with researchers to harness TPUs at scale.

Role involves strong ownership, stakeholder influence, and hands-on work with ML design, ML infrastructure, and performance tuning across GPU/TPU architectures.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 8+ years of software development experience, with 5+ years in ML/AI domains.
• 5+ years designing and deploying ML models and ML infrastructure.

Responsibilities

• Identify and maintain ML training and serving benchmarks representative of Google production.
• Drive performance improvements for ML workloads on TPUs and GPUs.
• Collaborate with product teams and researchers to onboard new ML models on TPU hardware and optimize for large-scale training.
• Analyze performance metrics to identify bottlenecks and implement scalable solutions.
• Engage with cross-functional teams to push core ML infrastructure and compiler/runtime optimizations.

Skills

Software dev

ML experience

Speech/Audio

Reinforcement learning

ML infrastructure

GPU acceleration

Education

Bachelor's degree or equivalent practical experience

Master’s degree or PhD in Engineering/CS

Tools

TensorFlow/TPU

OpenXLA

MLIR

CUDA

Job description

Google is seeking a Staff Software Engineer, TPU, Performance to advance ML performance on TPU systems and contribute to Gemini and OSS ML models. You will work on optimizing model architectures, tuning compilers and runtime for large-scale ML workloads, and collaborating with researchers to harness TPUs at scale.

Role involves strong ownership, stakeholder influence, and hands-on work with ML design, ML infrastructure, and performance tuning across GPU/TPU architectures.