Senior Software Engineer - Distributed AI & Cloud Infra

Google; Sunnyvale (CA); On-site; USD 174,000 - 252,000; Full time

Job summary

Google is seeking software engineers to advance next-generation technologies that scale to billions of users. The role involves designing and operating globally distributed systems, building scheduling and token‑oriented infrastructure for AI workloads, and ensuring high availability.

You will work with teams across Cloud and AI initiatives, contributing to secure, efficient, and scalable solutions. The position emphasizes leadership, cross‑functional collaboration, and the ability to tackle

Qualifications

• Bachelor's degree or equivalent practical experience.
• 5 years of experience with software development in one or more programming languages.
• 3 years of experience testing, maintaining, or launching software products, and 1 year of experience with software design and architecture.
• 3 years of experience with developing large-scale infrastructure, distributed systems or networks, or experience with compute technologies, storage or hardware architecture.

Responsibilities

• Design and operate globally distributed systems paths that handle millions of queries across multiple regions.
• Build distributed scheduling, dynamic reservation, and fair-sharing tuned to how LLMs actually behave - token-by-token streaming, bursty agent loops, speculative decoding, and prompt caching.
• Build self-healing systems with solid failover, fast coordination, and the telemetry needed to hit five-nines (99.999%) availability.
• Set the architectural direction for token orchestration and distribution across Cloud.

Skills

Data structures & algorithms

Generative AI

Distributed systems

Leadership

System design

Education

Bachelor's degree or equivalent practical experience

Master's degree or PhD in Computer Science or related field

Tools

Java

C/C++

Compute/storage hardware knowledge

Job description

Google is seeking software engineers to advance next-generation technologies that scale to billions of users. The role involves designing and operating globally distributed systems, building scheduling and token‑oriented infrastructure for AI workloads, and ensuring high availability.

You will work with teams across Cloud and AI initiatives, contributing to secure, efficient, and scalable solutions. The position emphasizes leadership, cross‑functional collaboration, and the ability to tackle