Software Development Engineer II, AWS SageMaker AI

Amazon Web Services (AWS); Bellevue (WA); On-site; USD 144,000 - 194,000; Full time

Benefits offered by this job

RSUs

Health insurance

401(k) matching

Paid time off

Job summary

Amazon.com Services LLC in Bellevue, WA is seeking a skilled software engineer to design, build, test, and operate services that manage foundation-model data preparation, training, evaluation, and deployment as reliable workflows.

You will own components end-to-end, from design to deployment, with heavy emphasis on distributed systems, scalability, and orchestration across SageMaker, EMR, and Kubernetes. Strong OO design and experience with C/C++/Java/Perl are expected.

Qualifications

• 3+ years of non-internship professional software development experience
• 2+ years of non-internship design or architecture experience
• 1+ years of software development engineer or related occupational experience
• 1+ years of designing and developing large-scale, multi-tiered, distributed software applications, tools, systems and services
• 1+ years of OO design experience

Responsibilities

• Design, build, test, and operate services that orchestrate foundation-model data preparation, training, evaluation, and deployment as contract-validated workflows
• Own delivery of components end-to-end from design through deployment, monitoring and on-call operations
• Build and extend compute-backend integrations and job launchers for large-scale training across SageMaker, EMR, AWS Batch and Kubernetes/EKS
• Improve platform resiliency and operability for long-running distributed jobs with checkpoint/resume, fault detection and retries
• Contribute to SDK, workflow orchestration, and CDK infrastructure used to deploy the platform
• Integrate containerized training and evaluation frameworks into the platform’s task and recipe model
• Contribute to design discussions, write technical designs, and uphold engineering best practices
• Collaborate with ML scientists and customers to translate training requirements into platform capabilities

Skills

Software development

Distributed systems

OO design

C/C++/Java/Perl

Education

Bachelor's degree in CS/Engineering/Math

Tools

Kubernetes/EKS

Airflow/Step Functions

AWS CDK/CloudFormation

PyTorch/FSDP

Job description

Description

At AWS SageMaker AI, we’re making it easy to build state-of-the-art foundation models on the cloud. Model Factory is our platform for building, training, customizing, and evaluating foundation models at scale. Instead of hand-chaining data prep, distributed training, evaluation, and deployment across thousands of GPU and AWS Trainium devices, Model Factory lets teams express the whole lifecycle as a single, contract-validated workflow — orchestrated, reproducible, and fully managed. As LLMs and Generative AI scale, Model Factory is the platform that turns frontier training research into a reliable, repeatable pipeline for our internal teams and customers.

Key job responsibilities

• Design, build, test, and operate services that orchestrate foundation-model data preparation, training, evaluation, and deployment as reliable, contract-validated workflows.
• Own delivery of individual components end-to-end — from design and implementation through deployment, monitoring, and on-call operations.
• Build and extend compute-backend integrations and job launchers — submitting, monitoring, and recovering large-scale training jobs across SageMaker (Training/Processing/HyperPod), EMR, AWS Batch, and Kubernetes/EKS.
• Improve the platform’s resiliency and operability for long-running distributed jobs — checkpoint/resume, fault detection and recovery, retries, and observability (metrics, logging, experiment tracking).
• Contribute to the SDK, workflow orchestration, and schema/contract layer that teams use to declare and run jobs, and to the CDK infrastructure that deploys the platform.
• Integrate containerized training and evaluation frameworks (e.g., PyTorch/FSDP, verl, NeMo/Megatron) into the platform’s task and recipe model.
• Contribute to design and architecture discussions, write clear technical designs, and uphold engineering best practices (code review, testing, operational readiness).
• Collaborate with ML scientists and internal customers to translate training requirements into reliable, self‑service platform capabilities, and help onboard and mentor interns and new engineers as you grow.

Basic Qualifications

• 3+ years of non-internship professional software development experience
• 2+ years of non-internship design or architecture (design patterns, reliability and scaling) of new and existing systems experience
• 1+ years of software development engineer or related occupational experience
• 1+ years of designing and developing large‑scale, multi-tiered, multi-threaded, embedded or distributed software applications, tools, systems, and services using: C#, C++, Java, or Perl experience
• 1+ years of Object Oriented Design experience
• Bachelor’s degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field
• Experience programming with at least one software programming language

Preferred Qualifications

• 3+ years of full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations experience
• Bachelor’s degree in computer science or equivalent
• Experience with workflow/pipeline orchestration (Airflow, Step Functions, or similar) and event-driven or service-oriented architectures.
• Experience with container and cluster compute (Kubernetes/EKS, Ray, Slurm, AWS Batch) and cloud infrastructure-as-code (AWS CDK/CloudFormation).
• Experience building and operating fully‑managed cloud services at scale, including resiliency, checkpointing, and fault tolerance for long-running jobs.
• Familiarity with machine-learning / deep-learning training workflows, GPU/accelerator compute (SageMaker HyperPod, AWS Trainium, P5-class GPUs), or distributed‑training frameworks (PyTorch FSDP, Megatron‑LM, DeepSpeed, verl).

Amazon is an equal opportunity employer and does not discriminate on the basis of protected veteran status, disability, or other legally protected status.

Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and hiring process, including support for the interview or onboarding process, please visit https://amazon.jobs/content/en/how-we-hire/accommodations for more information. If the country/region you’re applying in isn’t listed, please contact your Recruiting Partner.

The base salary range for this position is listed below. Your Amazon package will include sign‑on payments and restricted stock units (RSUs). Final compensation will be determined based on factors including experience, qualifications, and location. Amazon also offers comprehensive benefits including health insurance (medical, dental, vision, prescription, Basic Life & AD&D insurance and option for Supplemental life plans, EAP, Mental Health Support, Medical Advice Line, Flexible Spending Accounts, Adoption and Surrogacy Reimbursement coverage), 401(k) matching, paid time off, and parental leave. Learn more about our benefits at https://amazon.jobs/en/benefits.

USA, WA, Bellevue - 143,700.00 - 194,400.00 USD annually

Company

Amazon.com Services LLC

Job ID: A10492392