Software Engineering Manager, AI/ML Infrastructure and Performance Engineering

Google

Bengaluru

On-site

INR 3,500,000 - 5,500,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Google LLC in Bengaluru seeks an experienced Software Engineering Manager to lead AI/ML infrastructure and performance engineering. You will manage a team, shape product strategy, and drive performance across ML workloads and hardware/software stacks.

The role emphasizes building scalable ML systems, collaborating across teams, and driving efficiency in an enterprise-scale environment. Strong leadership and architectural skills are essential.

Qualifications

  • Bachelor's degree or equivalent practical experience.
  • 8 years of experience with software engineering, machine learning infrastructure, computer architecture, distributed computing, people management, debugging tools, communication.
  • Experience with people management, and building infrastructure to improve performance of ML systems and applications.

Responsibilities

  • Learn and build an intuitive understanding of existing data collection, analysis, and visualization workflows with deep introspection across Frameworks, Accelerated Linear Algebra (XLA) and runtime stack.
  • Support new and exciting ML paradigms by making contributions across the end-to-end stack and analysis tools; partner with ML Stack leads to understand model optimization use cases.
  • Work with OSS ML inference frameworks such as vLLM, SGLang to provide insights in Xprof about performance improvement opportunities.
  • Partner with other teams that own various parts of the ML stack to understand performance optimization use cases; work with OSS ML inference frameworks such as TorchTPU, vLLM, SGLang to provide insights into performance bottlenecks.

Skills

People management
Distributed computing
ML infrastructure
Debugging tools
Communication

Education

Bachelor's degree

Tools

TensorFlow
PyTorch
JAX
Keras
vLLM
SGLang

Job description

Software Engineering Manager, AI/ML Infrastructure and Performance Engineering

Share Software Engineering Manager, AI/ML Infrastructure and Performance Engineering

  • Bachelor's degree or equivalent practical experience.
  • 8 years of experience with software engineering, machine learning infrastructure, computer architecture, distributed computing, people management, debugging tools, communication.
  • Experience with people management, and building infrastructure to improve performance of Machine Learning (ML) systems and applications.
Preferred qualifications:
  • Experience with Performance Optimization, Graphics Processing Unit (GPU) Programming, High Performance Computing, Large Language Model, Open Source Contributor.
  • Experience with the Machine Learning infra and frameworks and with GPU or Tensor Processing Unit (TPU) performance analysis.
  • Experience in building agentic workflows for performance debugging and optimization. Ability to generate ideas and resolve ambiguity.
  • Experience with Machine Learning (ML) frameworks such as TensorFlow, JAX, and PyTorch, Keras. Experience with ML Inference frameworks such as vLLM, SG Lang, Pathways and in open-source software development, including experience in releasing and supporting open-source projects.
About the job

Like Google's own ambitions, the work of a Software Engineer goes beyond just Search. Software Engineering Managers have not only the technical expertise to take on and provide technical leadership to major projects, but also manage a team of Engineers. You not only optimize your own code but make sure Engineers are able to optimize theirs. As a Software Engineering Manager you manage your project goals, contribute to product strategy and help develop your team. Teams work all across the company, in areas such as information retrieval, artificial intelligence, natural language processing, distributed computing, large-scale system design, networking, security, data compression, user interface design; the list goes on and is growing every day. Operating with scale and speed, our exceptional software engineers are just getting started -- and as a manager, you guide the way.
With technical and leadership expertise, you manage engineers across multiple teams and locations, a large product budget and oversee the deployment of large-scale projects across multiple sites internationally.

Our team is focused around driving continuous improvements to the machine learning software/hardware stacks through providing insightful performance debugging for workloads and custom kernels. We provide insights by summarizing different views of captured profile data - such as trace timelines, memory usage, Compiler profiles, ML graph summaries

The ML, Systems, & Cloud AI (MSCA) organization at Google designs, implements, and manages the hardware, software, machine learning, and systems infrastructure for all Google services (Search, YouTube, etc.) and Google Cloud. Our end users are Googlers, Cloud customers and the billions of people who use Google services around the world.

We prioritize security, efficiency, and reliability across everything we do - from developing our latest TPUs to running a global network, while driving towards shaping the future of hyperscale computing. Our global impact spans software and hardware, including Google Cloud’s Vertex AI, the leading AI platform for bringing Gemini models to enterprise customers.

Responsibilities
  • Learn and build an intuitive understanding of existing data collection, analysis, and visualization workflows with deep introspection across Frameworks, Accelerated Linear Algebra (XLA) and runtime stack.
  • Support new and exciting ML paradigms (such as horizontal scaling for upcoming TPU chips) by making contributions across the end to end stack and analysis tools. Partner with ML Stack leads to understand model optimization use cases and bring debugging to feel native in 3P environments (VSCode, Cursor, Grafana, etc.).
  • Work with OSS ML inference frameworks such as vLLM, SGLang to provide insights in Xprof about performance improvement opportunities.
  • Partner with other teams that own various parts of the ML stack to understand performance optimization use cases. Work with OSS ML inference frameworks such as TorchTPU, vLLM, SGLang to provide insights into performance bottlenecks.

Google is proud to be an equal opportunity and affirmative action employer. We are committed to building a workforce that is representative of the users we serve, creating a culture of belonging, and providing an equal employment opportunity regardless of race, creed, color, religion, gender, sexual orientation, gender identity/expression, national origin, disability, age, genetic information, veteran status, marital status, pregnancy or related condition (including breastfeeding), expecting or parents-to-be, criminal histories consistent with legal requirements, or any other basis protected by law. See also Google's EEO Policy , Know your rights: workplace discrimination is illegal , Belonging at Google , and How we hire .

Google is a global company and, in order to facilitate efficient collaboration and communication globally, English proficiency is a requirement for all roles unless stated otherwise in the job posting.

Equity is granted exclusively and discretionarily by Alphabet Inc. on the basis of an agreement concluded between you and Alphabet Inc. Alphabet Inc. is your sole contractual partner with respect to equity grants. GSU grants are not guaranteed, are discretionary, are subject to approval by the Alphabet Inc. board of directors or its delegate, the terms of the relevant Alphabet Inc. stock plan, and your grant agreement. They have no impact on statutory payments. Current or past grants do not confer an acquired right.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Software Engineering Manager, Platforms Infrastructure Engineering
Senior Software Engineering Manager, Platforms Infrastructure Engineering

Google • Bengaluru

On-site
INR 4,000,000 - 7,000,000
Staff Software Engineer, Search ML Infrastructure
Staff Software Engineer, Search ML Infrastructure

Google LLC • Bengaluru

On-site
INR 3,800,000 - 7,000,000
Software Engineer Manager, Google Cloud Business Platform
Software Engineer Manager, Google Cloud Business Platform

Google • Bengaluru

On-site
INR 4,200,000 - 6,800,000
Senior Software Engineer, ML Compiler, Frameworks and Performance
Senior Software Engineer, ML Compiler, Frameworks and Performance

Google LLC • Bengaluru

On-site
INR 4,000,000 - 8,000,000
Software Engineering Manager, Search
Software Engineering Manager, Search

Google LLC • Bengaluru

On-site
INR 2,600,000 - 3,800,000
Senior Software Engineer, Search AI Infra
Senior Software Engineer, Search AI Infra

Google Inc • Bengaluru

On-site
INR 2,800,000 - 4,600,000
Staff Software Engineer, Search Production Intelligence
Staff Software Engineer, Search Production Intelligence

Google • Bengaluru

On-site
INR 4,000,000 - 7,000,000
Tech Lead Manager, Data Protection Understanding, AI Acceleration, Core
Tech Lead Manager, Data Protection Understanding, AI Acceleration, Core

Google LLC • Bengaluru

On-site
INR 4,200,000 - 5,600,000
Software Engineer, Security, Privacy, Detection
Software Engineer, Security, Privacy, Detection

Google LLC • Hyderabad

On-site
INR 400,000 - 800,000
Staff Software Engineer, Google Global Infrastructure
Staff Software Engineer, Google Global Infrastructure

Google • Bengaluru

On-site
INR 6,000,000 - 9,000,000