Lead Architect, AI Inference on GKE

Google; Sunnyvale (CA); On-site; USD 307,000 - 427,000; Full time

Benefits offered by this job

Health benefits

401(k) with company match

Paid time off

Sick time

Maternity leave

Baby bonding leave

Holidays

Job summary

Google Cloud's Kubernetes Engine (GKE) team seeks a Principal Engineer to reinvent the platform for AI inference at scale. You will lead architectural reinvention, defining high-performance scheduling, multi-host TPU/GPU coordination, and accelerator orchestration across the GKE fleet.

You will drive strategic priorities for integrating llm-d and AI-first roadmaps, collaborating with model builders and the OSS community, and shaping industry standards.

Qualifications

• Bachelor's degree in Computer Science, a related technical field, or equivalent practical experience.
• 15 years of experience in software engineering, or 15 years of experience with an advanced degree.
• Experience building distributed systems and driving technical strategy for platform-level infrastructure.
• Experience with Kubernetes, container runtimes, and AI/ML infrastructure (e.g., inference serving, LLM, hardware accelerators).

Responsibilities

• Lead the architectural direction for llm-d, ensuring a scalable foundation for distributed LLM and RL serving across the GKE fleet.
• Define GKE's evolution to support massive-scale inference and RL with high-throughput networking.
• Partner with AI model builders to co-develop an AI-first roadmap leveraging Google's silicon to optimize throughput.
• Lead OSS contributions to establish standards for AI, RL, and accelerator orchestration.

Skills

Distributed systems

Kubernetes

AI/ML infrastructure

Container runtimes

Education

Bachelor's degree in CS or related field

Tools

Kubernetes

Container runtimes

LLM infrastructure

Job description

Google Cloud's Kubernetes Engine (GKE) team seeks a Principal Engineer to reinvent the platform for AI inference at scale. You will lead architectural reinvention, defining high-performance scheduling, multi-host TPU/GPU coordination, and accelerator orchestration across the GKE fleet.

You will drive strategic priorities for integrating llm-d and AI-first roadmaps, collaborating with model builders and the OSS community, and shaping industry standards.