Senior Software Engineer, AI/LLM on GDC Platform

Google; Sunnyvale (CA); On-site; USD 174,000 - 252,000; Full time

Job summary

Google is seeking a Senior Software Engineer for the Google Distributed Cloud AI team in Sunnyvale, CA. The role focuses on building scalable software for LLM inference, with emphasis on model lifecycle, data loading, routing, and load balancing, as well as integration with platform services like billing, logging, and security.

The candidate should have strong Go experience and background in large-scale infrastructure, with opportunities to mentor junior engineers and work across hybrid cloud

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 5 years of experience with software development in one or more programming languages.
• 3 years of experience testing, maintaining, or launching software products, and 1 year of experience with software design and architecture.
• 3 years of experience with developing large-scale infrastructure, distributed systems or networks, or experience with compute technologies, storage or hardware architecture.
• Experience programming in Go for software development, including AI/ML applications.

Responsibilities

• Lead the technical design, development, and optimization of software components critical for Large Language Model (LLM) inference serving on GDC.
• Drive horizontal integration across core platform services, including billing, logging, observability, security, and quota management.
• Implement and enhance serving capabilities to support advanced LLM techniques like disaggregated serving, speculative decoding, quantization, and efficient model sharding across distributed hardware.
• Collaborate closely with internal teams developing core LLM frameworks, container orchestration (Kubernetes, Google Kubernetes Engine (GKE)), networking infrastructure, and hardware acceleration to build a cohesive and high-performance serving platform.

Skills

Go

Distributed systems

Large-scale infrastructure

AI/ML applications

Mentoring

Education

Bachelor's degree or equivalent practical experience

Master's degree or PhD in Computer Science or related field

Tools

Kubernetes

GKE

Job description

Google is seeking a Senior Software Engineer for the Google Distributed Cloud AI team in Sunnyvale, CA. The role focuses on building scalable software for LLM inference, with emphasis on model lifecycle, data loading, routing, and load balancing, as well as integration with platform services like billing, logging, and security.

The candidate should have strong Go experience and background in large-scale infrastructure, with opportunities to mentor junior engineers and work across hybrid cloud