GKE AI Inference Platform Architect

Google Inc.; Seattle (WA); On-site; USD 307,000 - 427,000; Full time

Benefits offered by this job

Health insurance

401(k) with company match

Paid time off

Job summary

Google is seeking a Principal Engineer for the GKE Platform handling AI inference workloads. You will drive architectural reinvention of GKE to support massive-scale AI inference, RL serving, and integration with llm-d while collaborating across Google Cloud and the OSS community.

Expect leadership in high-scale orchestration, diverse hardware (TPU/GPU), and partnerships with model builders to optimize throughput and density. Strong emphasis on equity and global collaboration.

Qualifications

• Bachelor's degree in Computer Science, a related technical field, or equivalent practical experience.
• 15 years of experience in software engineering, or 15 years of experience with an advanced degree.

Responsibilities

• Lead the architectural direction for llm-d, ensuring a highly optimized, scalable foundation for distributed LLM and RL serving across the GKE fleet.
• Define GKE's evolution to support massive-scale inference and RL, solving novel orchestration problems in dynamic resource allocation, multi-host TPU/GPU scheduling, and high-throughput networking.
• Partner with strategic AI model builders, DeepMind, and Vertex AI to co-develop an AI-first roadmap, leveraging Google's custom silicon to optimize throughput and compute density.
• Lead the broader Kubernetes ecosystem and OSS community, driving key upstream initiatives to establish industry standards for AI, RL, and accelerator orchestration.

Skills

Kubernetes

AI/ML infrastructure

Distributed systems

Cross-functional collaboration

Tools

Container runtimes

Job description

Google is seeking a Principal Engineer for the GKE Platform handling AI inference workloads. You will drive architectural reinvention of GKE to support massive-scale AI inference, RL serving, and integration with llm-d while collaborating across Google Cloud and the OSS community.

Expect leadership in high-scale orchestration, diverse hardware (TPU/GPU), and partnerships with model builders to optimize throughput and density. Strong emphasis on equity and global collaboration.