Senior AI/ML Systems Architect

Google; Mountain View (CA); On-site; USD 207,000 - 300,000; Full time

Job summary

Google is seeking expert software engineers to advance AI-driven workloads across large-scale platforms. You will own architecture, design, and delivery of high-performance ML pipelines, model compression, and efficient inference serving.

The role emphasizes translating executive mandates into concrete, efficiency-driven engineering tasks. The team focuses on minimizing latency and improving throughput while integrating cutting-edge ML tools across Google Cloud workloads.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 8 years of experience in software development.
• 5 years of experience testing, launching software products, and 3 years of experience with software design and architecture.
• 5 years of experience with one or more of: speech/audio, reinforcement learning, ML infrastructure, or specialization in another ML field.
• 5 years of experience with ML design and ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• Experience integrating generative AI tools or LLM interfaces into workflows.

Responsibilities

• Lead technical architecture and system design for complex 1–6-month Forward Deployed Engineer embeds and 2–4-week Strike Sprints targeting high-leverage ML bottlenecks.
• Design, prototype, and write robust production C++ and Python code for model compression, speculative decoding engines, dynamic batching layers, and high-throughput serving pipelines.
• Diagnose distributed latency and throughput bottlenecks across clusters, implementing low-level kernel and memory optimizations (XLA, Pallas, Custom Ops).
• Translate nebulous VP-level mandates into rigorous efficiency scopes with latency, FLOPs, and tokenomics thresholds.
• Design automated distillation, pruning, and quantization pipelines that convert large foundation models into compact, efficient student models.

Skills

Software development experience

ML design & infrastructure

Speech/Audio / RL / ML infra

Software architecture & design

Generative AI / LLM integration

Education

Bachelor’s degree or equivalent practical experience

Master’s degree or PhD in Engineering/CS or related field

Tools

XLA

Pallas

CUDA

Triton

Custom TPU kernels

Job description

Google is seeking expert software engineers to advance AI-driven workloads across large-scale platforms. You will own architecture, design, and delivery of high-performance ML pipelines, model compression, and efficient inference serving.

The role emphasizes translating executive mandates into concrete, efficiency-driven engineering tasks. The team focuses on minimizing latency and improving throughput while integrating cutting-edge ML tools across Google Cloud workloads.