Senior TPU Performance Co-Design Engineer (LLM Serving)

Google; Sunnyvale (CA); On-site; USD 174,000 - 252,000; Full time

Job summary

Google Cloud is seeking a Senior Performance Co-Design Engineer for TPU work in Sunnyvale to optimize serving performance of large language models on custom AI silicon. You will analyze current and emerging models, build tools to probe bottlenecks, and influence architecture choices with hardware and software partners.

The role centers on LLM inference latency and throughput, collaborating with researchers and engineers to push the frontier of enterprise AI on Google Cloud infrastructure, with a

Qualifications

• Bachelor’s degree in CS/EE/CE or equivalent practical experience.
• 5 years in performance modeling/engineering, computer architecture, co-design, or systems engineering.
• Experience programming in C++ or Python.

Responsibilities

• Conduct comprehensive serving performance studies on current and emerging LLMs.
• Develop and maintain advanced simulation, profiling, and modeling tools to identify bottlenecks and project workload performance.
• Partner with model researchers, software and hardware teams to co-design architectural improvements for LLM inference latency and throughput.
• Drive data-backed decisions that influence the roadmap for TPU/Cloud Silicon architectures.

Skills

Performance modeling

Computer architecture

Co-design

C++

Python

Education

Bachelor's degree or equivalent practical experience

Master's degree or PhD (preferred)

Tools

Profiling tools

Job description

Google Cloud is seeking a Senior Performance Co-Design Engineer for TPU work in Sunnyvale to optimize serving performance of large language models on custom AI silicon. You will analyze current and emerging models, build tools to probe bottlenecks, and influence architecture choices with hardware and software partners.

The role centers on LLM inference latency and throughput, collaborating with researchers and engineers to push the frontier of enterprise AI on Google Cloud infrastructure, with a