Principal Engineer, AI Inference Platform on GKE

Google; Seattle (WA); On-site; USD 307,000 - 427,000; Full time

Benefits offered by this job

Health insurance

Dental insurance

Vision insurance

Life insurance

Disability insurance

401(k) with company match

Paid time off 20 days/year

Job summary

Google Kubernetes Engine (GKE) is advancing to become the inference engine for the world, enabling massive-scale AI workloads and specialized scheduling across the GKE fleet. The Principal Engineer will set long-term strategic priorities for integrating high-scale AI inference and llm-d into the platform.

This role leads collaboration with DeepMind, Vertex AI, and upstream OSS projects, driving the technical vision while ensuring scalable, performance-focused infrastructure for AI deployments at

Qualifications

• Bachelor's degree in Computer Science or related field, or equivalent practical experience.
• 15 years of software engineering experience or 15 years with an advanced degree.
• Experience building distributed systems and driving technical strategy for platform-level infrastructure.
• Experience with Kubernetes, container runtimes, and AI/ML infrastructure (e.g., inference serving, LLM, hardware accelerators).

Responsibilities

• Define architectural direction for llm-d to support distributed LLM and RL serving across the GKE fleet.
• Evolve GKE to support massive-scale inference and RL, addressing resource allocation and multi-host scheduling.
• Collaborate with AI model builders and Vertex AI on an AI-first roadmap, leveraging hardware accelerators.
• Lead OSS contributions and establish industry standards for AI, RL, and orchestration.

Skills

Distributed systems

Technical strategy

LLM/AI inference

High-performance networking

Education

Bachelor's degree in CS or related field

Master's degree or PhD (preferred)

Tools

Kubernetes

Container runtimes

AI/ML infrastructure

Job description

Google Kubernetes Engine (GKE) is advancing to become the inference engine for the world, enabling massive-scale AI workloads and specialized scheduling across the GKE fleet. The Principal Engineer will set long-term strategic priorities for integrating high-scale AI inference and llm-d into the platform.

This role leads collaboration with DeepMind, Vertex AI, and upstream OSS projects, driving the technical vision while ensuring scalable, performance-focused infrastructure for AI deployments at