Staff Software Engineer, AI Engines 3P TPU Inference

Google Inc.; Mountain View (CA); On-site; USD 207,000 - 300,000; Full time

Job summary

Google seeks a Staff Software Engineer for AI Engines 3P TPU Inference in Mountain View, CA. You will work on ML infrastructure, compilers, and runtimes, guiding engineering decisions for scalable ML systems and collaborating across teams to advance TPU-driven AI workloads.

The role emphasizes leadership, cross-functional collaboration, and delivering high-impact ML capabilities in a fast-paced environment. A strong background in ML infrastructure and systems design is essential.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 8 years of experience in software development.
• 5 years testing and launching software products, 3 years in design/architecture.
• 5 years of experience in one or more: Speech/Audio, reinforcement learning, ML infrastructure, or related ML field.

Responsibilities

• Exercise judgment to guide engineering choices for ML systems at scale.
• Innovate directions for infrastructure over a 12-month horizon amid changing tech.
• Deliver impactful capabilities to Cloud and partner product areas.
• Migrate TensorFlow/JAX/PyTorch runtimes and workflows onto ML Runtime with minimal user disruption.
• Partner with teams to transfer innovations into products.

Skills

Software development

ML design

Reinforcement learning

Speech/Audio

Model deployment

Education

Bachelor's degree or equivalent practical experience

Master's or PhD (preferred)

Tools

TensorFlow

JAX

PyTorch

TFRT

PJRT

TPU system design

Job description

Staff Software Engineer, AI Engines 3P TPU Inference

corporate_fare Google place Mountain View, CA, USA

• Bachelor’s degree or equivalent practical experience.
• 8 years of experience in software development.
• 5 years of experience testing, and launching software products, and 3 years of experience with software design and architecture.
• 5 years of experience with one or more of the following: Speech/audio (e.g., technology duplicating and responding to the human voice), reinforcement learning (e.g., sequential decision making), ML infrastructure, or specialization in another ML field.
• 5 years of experience with ML design and ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).

Preferred qualifications:

• Master’s degree or PhD in Engineering, Computer Science, or a related technical field.
• 8 years of experience with data structures and algorithms.
• 3 years of experience in a technical leadership role leading project teams and setting technical direction.
• 3 years of experience working in a complex, matrixed organization involving cross-functional, or cross-business projects.
• Experience with TPUs, TPU system design, and GPUs.
• Expertise in ML compilers and runtimes.

About the job

Google's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale, and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking and data storage, security, artificial intelligence, natural language processing, UI design and mobile; the list goes on and is growing every day. As a software engineer, you will work on a specific project critical to Google’s needs with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, display leadership qualities and be enthusiastic to take on new problems across the full-stack as we continue to push technology forward.

With your technical expertise you will manage project priorities, deadlines, and deliverables. You will design, develop, test, deploy, maintain, and enhance software solutions.

As a Staff Software Engineer, you will have expertise in ML compilers and runtimes. You will have management and leadership experience and strong partnership skills to enact impact leveraging cross-organizational technical collaborations.

This organization provides ML infrastructure for all of Google. In close partnership with Research, this organization delivers the Gemini models, and drives their usage within and outside Google. Within this organization, the Machine Learning Infrastructure Runtime and API teams mission is to deliver highly performant, modular and scalable ML software infrastructure.

The Google Cloud AI Research team addresses AI challenges motivated by Google Cloud’s mission of bringing AI to tech, healthcare, finance, retail and many other industries. We work on a range of unique problems focused on research topics that maximize scientific and real-world impact, aiming to push the state-of-the-art in AI and share findings with the broader research community. We also collaborate with product teams to bring innovations to real-world impact that benefits our customers.

The US base salary range for this full-time position is $207,000-$300,000 + bonus + equity + benefits. Our salary ranges are determined by role, level, and location. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training. Your recruiter can share more about the specific salary range for your preferred location during the hiring process.

Please note that the compensation details listed in US role postings reflect the base salary only, and do not include bonus, equity, or benefits. Learn more about benefits at Google .

Responsibilities

• Exercise judgment to guide sustainable engineering choices for ML systems at scale.
• Innovate next directions for infrastructure over a 12-month time horizon given a rapidly changing technology landscape.
• Deliver impactful capability and optimization impact to Cloud and partner product areas.
• Gracefully and incrementally migrate existing frameworks (TensorFlow, JAX, PyTorch) runtimes (TF Executor, TFRT, PJRT) and product areas custom workflows (AdBrain) onto ML Runtime, minimizing any user disruption.
• Partner with GDM to transfer key innovations into products developed by your team and partner teams.

Google is proud to be an equal opportunity and affirmative action employer. We are committed to building a workforce that is representative of the users we serve, creating a culture of belonging, and providing an equal employment opportunity regardless of race, creed, color, religion, gender, sexual orientation, gender identity/expression, national origin, disability, age, genetic information, veteran status, marital status, pregnancy or related condition (including breastfeeding), expecting or parents-to-be, criminal histories consistent with legal requirements, or any other basis protected by law. See also Google's EEO Policy , Know your rights: workplace discrimination is illegal , Belonging at Google , and How we hire .

Google is a global company and, in order to facilitate efficient collaboration and communication globally, English proficiency is a requirement for all roles unless stated otherwise in the job posting.