Software Development Engineer, AI Runtime

Amazon Inc.; Pune District; On-site; INR 2,800,000 - 5,200,000; Full time

Job summary

Amazon Inc. is hiring a Software Development Engineer for AI Runtime in Pune to design and scale the next generation of Alexa+ runtime infrastructure. You will build Tier-1 distributed systems, orchestrate LLM inference, and ensure low-latency delivery across devices worldwide.

You will mentor teammates, collaborate with ML teams, and drive architectural decisions balancing performance, reliability, and cost. Applicants should have strong DS skills and hands-on coding experience.

Qualifications

• 3+ years of non-internship professional software development experience.
• 2+ years design/architecture of distributed systems.
• Bachelor's degree or foreign equivalent in CS/Engineering/Math.
• Experience programming with at least one language.

Responsibilities

• Design, build, and operate large-scale distributed services that power Alexa+ Expert executions.
• Develop and optimize LLM orchestration pipelines — including model serving, prompt routing, and inference optimization.
• Build multi-agent coordination systems that route requests to the right expert agents with minimal latency.
• Own the operational excellence of Tier-1 services, ensuring 99.99%+ availability at scale.
• Collaborate with applied scientists and ML engineers to integrate new AI capabilities into production.
• Drive architectural decisions that balance performance, reliability, and cost efficiency.
• Mentor teammates and raise the engineering bar through code reviews, design reviews, and technical leadership.

Skills

Software development

System design

Programming languages

Education

Bachelor's degree in CS/Engineering/Math

Foreign equivalents accepted

Job description

Software Development Engineer, AI Runtime

We are looking for a passionate and experienced Software Development Engineer to help us build and scale the next generation of AI runtime infrastructure for Alexa+. You will design and implement Tier-1 distributed systems that orchestrate expert agent executions, manage LLM inference pipelines, and ensure reliable, low-latency delivery across Alexa-enabled devices globally.

Key job responsibilities

• Design, build, and operate large-scale distributed services that power Alexa+ Expert executions
• Develop and optimize LLM orchestration pipelines — including model serving, prompt routing, and inference optimization
• Build multi-agent coordination systems that route requests to the right expert agents with minimal latency
• Own the operational excellence of Tier-1 services, ensuring 99.99%+ availability at scale
• Collaborate with applied scientists and ML engineers to integrate new AI capabilities into production
• Drive architectural decisions that balance performance, reliability, and cost efficiency
• Mentor teammates and raise the engineering bar through code reviews, design reviews, and technical leadership

About the team

The AI Runtime team is the backbone of Alexa+. We own the infrastructure that powers Expert executions — the multi-agent systems that bring Alexa+'s intelligent capabilities to life for hundreds of millions of customers worldwide. Our services orchestrate LLM inference, coordinate expert agents, and deliver responses at massive scale with low latency. When a customer interacts with Alexa+ and it feels magical, our platform made it happen.

Basic Qualifications

• 3+ years of non-internship professional software development experience
• 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
• Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field
• Experience programming with at least one software programming language

Preferred Qualifications

• 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
• Bachelor's degree in computer science or equivalent
• System design with AI components and able to evaluate latency, cost, reliability, and accuracy.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status. Veterans, military spouses, and people with disabilities are encouraged to apply.