Senior LLM Serving Performance Co-Design Engineer

Google Inc.; Sunnyvale (CA); On-site; USD 174,000 - 252,000; Full time

Job summary

Google Cloud’s mission is to make every business successful through AI by combining cutting-edge technology, infrastructure, and talent. The TPU Chip Architecture and Performance Co-design team optimizes Google's custom AI silicon for next-generation ML models, including LLMs, on our own hardware.

As a Senior Performance Co-design Engineer, you will focus on LLM Serving Studies, analyzing and optimizing serving performance on our custom hardware, and collaborating with hardware architects to

Qualifications

• Bachelor’s degree in Computer Science, Electrical Engineering, Computer Engineering, a related field, or equivalent practical experience.
• 5 years of experience in performance modeling/engineering, computer architecture, co-design, or systems engineering.
• Experience programming in C++ or Python.

Responsibilities

• Conduct comprehensive serving performance studies on current and emerging LLMs, including 1P/3P models.
• Develop and maintain advanced simulation, profiling, and modeling tools to identify bottlenecks and project serving workload performance.
• Partner with model researchers, software and hardware teams to co-design architectural improvements for LLM inference latency and throughput.
• Drive data-backed decisions that influence the roadmap for future TPU/Cloud Silicon architectures.

Job description

Google Cloud’s mission is to make every business successful through AI by combining cutting-edge technology, infrastructure, and talent. The TPU Chip Architecture and Performance Co-design team optimizes Google's custom AI silicon for next-generation ML models, including LLMs, on our own hardware.

As a Senior Performance Co-design Engineer, you will focus on LLM Serving Studies, analyzing and optimizing serving performance on our custom hardware, and collaborating with hardware architects to