Staff Research Software Engineer, AI Inference & Efficiency

Google Inc.; Santo Niño 1st; On-site; PHP 10,880,000 - 16,815,000; Full time

Job summary

Google Research Singapore is advancing fundamental AI capabilities, pioneering Generative AI models, and leading a small agile team in a fast-paced environment. The role focuses on building scalable AI inference systems and collaborating across teams to publish research and drive innovations for billions of users.

You will design novel algorithms for efficient LLM inference, optimize low-level software, and work with hardware accelerators to push performance boundaries in AI deployment.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 8 years of experience in software development.
• 5 years of experience testing, and launching software products, and 3 years of experience with software design and architecture.
• 5 years of experience with one or more of the following: Speech/audio (e.g., technology duplicating and responding to the human voice), reinforcement learning (e.g., sequential decision making), ML infrastructure, or specialization in another ML field.
• 5 years of experience with ML design and ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• Experience integrating generative AI tools or LLM interfaces into workflows.

Responsibilities

• Design and implement novel algorithms that optimize the inference stack for LLMs to ensure the highest throughput and lowest latency for billions of users including sampling techniques, speculative decoding advancements, model compression, knowledge distillation and quantization strategies.
• Work on development and optimization of the low-level software that runs AI models efficiently. Implement and optimize custom kernels for inference, focusing on improving latency, throughput, and memory usage, across different hardware and model architectures.
• Co-design algorithms that exploit the unique architectural advantages of custom hardware and accelerators. Collaborate with hardware architects on next-generation systems and use a deep understanding of both model architecture and hardware to inform algorithm selection.
• Amplify impact and influence the research ecosystem through publications and scientific dissemination.

Education

Bachelor’s degree or equivalent practical experience.

8 years of experience in software development.

5 years of experience testing, and launching software products, and 3 years of experience with software design and architecture.

5 years of experience with one or more of the following: Speech/audio, reinforcement learning, ML infrastructure, or related ML field.

5 years of experience with ML design and ML infrastructure (deployment, evaluation, data processing, debugging, fine tuning).

Experience integrating generative AI tools or LLM interfaces into workflows.

Job description

Google Research Singapore is advancing fundamental AI capabilities, pioneering Generative AI models, and leading a small agile team in a fast-paced environment. The role focuses on building scalable AI inference systems and collaborating across teams to publish research and drive innovations for billions of users.

You will design novel algorithms for efficient LLM inference, optimize low-level software, and work with hardware accelerators to push performance boundaries in AI deployment.