AI Rapid Response Engineer — ML Efficiency & Systems

Socket.dev; Mountain View (CA); On-site; USD 207,000 - 300,000; Full time

Job summary

Google's software engineers develop the next-generation technologies that change how billions connect, explore, and interact with information and one another. We're looking for engineers who bring ideas from AI, large-scale systems, security, NLP, and more to push technology forward.

On the AI Rapid Response Team you will translate high-level mandates into efficient engineering solutions, write high-performance production code, and design model efficiency pipelines for Google's demanding ML

Qualifications

• 8 years of experience in software development.
• Bachelor’s degree or equivalent practical experience.
• 5 years of experience testing, and launching software products, and 3 years of experience with software design and architecture.
• 5 years of experience with ML design and ML infrastructure.
• Experience integrating generative AI tools or LLM interfaces into workflows.

Responsibilities

• Lead technical architecture and system design for complex embeds and sprints.
• Design, prototype, and write robust production C++ and Python code for model compression and high-throughput serving pipelines.
• Diagnose latency and throughput bottlenecks across ML clusters, implementing low-level kernel and memory optimizations.
• Deconstruct executive mandates into rigorous efficiency scopes with strict latency and FLOPs thresholds.
• Design automated distillation, pruning, and quantization pipelines for compact models.

Skills

Software development experience

ML design

ML infrastructure

Generative AI integration

Leadership

Education

Bachelor’s degree or equivalent practical experience

Master’s degree or PhD in Engineering/CS or related field

Tools

XLA

Pallas

CUDA

Triton

Custom TPU kernels

Job description

Google's software engineers develop the next-generation technologies that change how billions connect, explore, and interact with information and one another. We're looking for engineers who bring ideas from AI, large-scale systems, security, NLP, and more to push technology forward.

On the AI Rapid Response Team you will translate high-level mandates into efficient engineering solutions, write high-performance production code, and design model efficiency pipelines for Google's demanding ML