Senior ML Software Engineer, AI Accelerator & Inference

Amazon Web Services (AWS); Seattle (WA); On-site; USD 168,100 - 227,400; Full time

Job summary

Annapurna Labs (U.S.) Inc. in Seattle seeks a Senior Software Engineer to shape the evolution of AWS Neuron, AWS's AI accelerator stack. You will drive distributed AI across ML frameworks and hardware, focusing on high-performance inference for state-of-the-art models.

Join a team that bridges PyTorch and JAX with XLA, optimizes Trainium and Inferentia, and creates tools improving LLM accuracy and efficiency at scale.

Qualifications

• 5+ years of full software development lifecycle experience.
• 5+ years of programming experience with Python or C++ and PyTorch.
• Experience with AI acceleration via quantization, parallelism, model compression, batching, KV caching, vllm serving.
• Experience with accuracy debugging & tooling and performance benchmarking of AI accelerators.
• Fundamentals of ML and DL models, their architecture, training and inference lifecycles, plus optimization work.

Responsibilities

• Drive Evolution of Distributed AI at AWS Neuron, bridging ML frameworks and hardware.
• Architect the bridge between PyTorch, JAX and AI hardware; push scalable AI model execution.
• Spearhead benchmark methodologies and performance tuning for AI accelerators.
• Develop ML tools to improve LLM accuracy and efficiency.
• Collaborate with silicon architects to influence accelerator design.

Skills

Distributed systems

AI acceleration

Performance optimization

Debugging tooling

Inference optimization

Education

Bachelor's degree in CS or equivalent

Master's degree in CS or ML or equivalent

Tools

Python

C++

PyTorch

Job description

Annapurna Labs (U.S.) Inc. in Seattle seeks a Senior Software Engineer to shape the evolution of AWS Neuron, AWS's AI accelerator stack. You will drive distributed AI across ML frameworks and hardware, focusing on high-performance inference for state-of-the-art models.

Join a team that bridges PyTorch and JAX with XLA, optimizes Trainium and Inferentia, and creates tools improving LLM accuracy and efficiency at scale.