Senior AI Systems Engineer, LLM Infrastructure

Amazon; Palo Alto (CA); On-site; USD 193,000 - 262,000; Full time

Job summary

Amazon is seeking a Senior Software Development Engineer for Stores Foundational AI - Rufus in Palo Alto, CA. You will architect scalable ML infrastructure for LLM training, post-training workflows, and live-traffic data signals to improve customer experiences.

The role demands 5+ years in software development, strong systems design and leadership, and hands-on experience with ML platforms like vLLM, SGLang, TensorRT, and CUDA kernels.

Qualifications

• 5+ years of non-internship professional software development experience.
• 5+ years of programming with at least one software programming language experience.
• 5+ years of leading design or architecture (design patterns, reliability and scaling) of new and existing systems experience.
• Experience as a mentor, tech lead or leading an engineering team.
• Experience with vLLM, SGLang, TensorRT or similar platforms in production environments.
• Experience with CUDA kernels or ML/low-level kernels.
• 5+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience.
• Experience with Machine Learning and Large Language Model fundamentals, including architecture, training/inference lifecycles, and optimization of model execution

Responsibilities

• Architect and build scalable ML infrastructure powering LLM training and post-training workflows, including supervised fine-tuning, reinforcement learning, and continuous learning from live traffic
• Transform real-world customer interactions into high-quality training signals, enabling continuous model improvement and better customer experiences
• Build and optimize post-training and RL systems, including reward modeling, policy optimization, data collection loops.
• Drive experimentation and iteration velocity by building tooling and frameworks that enable rapid hypothesis testing, signal validation, and model quality improvements
• Partner closely with applied scientists to translate frontier techniques (e.g., RLHF, agentic workflows, multi-turn optimization) into reliable, production-grade systems
• Own systems end-to-end, including design, implementation, deployment, observability, and operational excellence
• Raise the engineering bar through technical leadership, design reviews, and mentorship, influencing best practices across the organization

Education

Bachelor's degree in computer science or equivalent

Tools

vLLM

SGLang

TensorRT

CUDA

Job description

Amazon is seeking a Senior Software Development Engineer for Stores Foundational AI - Rufus in Palo Alto, CA. You will architect scalable ML infrastructure for LLM training, post-training workflows, and live-traffic data signals to improve customer experiences.

The role demands 5+ years in software development, strong systems design and leadership, and hands-on experience with ML platforms like vLLM, SGLang, TensorRT, and CUDA kernels.