SageMaker ML Training Engineer II - Scale & Resilience

AI Chopping Block, Inc.; Bellevue, Northern (WA, KY); Hybrid; USD 144,000 - 194,000; Full time

Job summary

Amazon's SageMaker AI Training Job team is seeking a Software Development Engineer to help build and operate mission-critical systems at scale, enabling training jobs across hundreds of thousands of machines with minimal failure.

You will own components end-to-end, integrate containerized training frameworks, contribute to SDKs and CDK, and collaborate with ML scientists to translate requirements into self-service platform capabilities. This is a hands-on, high-impact role in Bellevue, WA.

Qualifications

• 3+ years of non-internship professional software development experience
• 2+ years of non-internship design or architecture experience for scalable systems
• 1+ years of software development engineer or related occupational experience
• 1+ years of designing and developing large-scale distributed software applications
• 1+ years of Object Oriented Design experience
• Bachelor's degree or foreign equivalent in Computer Science, Engineering, Mathematics, or related field
• Experience programming with at least one software programming language

Responsibilities

• Design, build, test, and operate services that orchestrate data preparation, training, evaluation, and deployment as reliable workflows
• Own delivery of components end-to-end—from design and implementation through deployment, monitoring, and on-call operations
• Build and extend compute-backend integrations and job launchers across SageMaker and AWS Batch
• Improve resiliency and operability for long-running distributed jobs—checkpoint/resume, fault detection and recovery, retries, observability
• Contribute to the SDK, workflow orchestration, and schema/contract layer and to CDK infrastructure
• Integrate containerized training and evaluation frameworks (e.g., PyTorch/FSDP, NeMo/Megatron) into the platform
• Contribute to design and architecture discussions and write clear technical designs
• Collaborate with ML scientists to translate training requirements into reliable platform capabilities

Skills

C++

Java

C#

Perl

OO Design

Python

Education

Bachelor's degree in Computer Science, Engineering, Mathematics, or related field

Tools

PyTorch

FSDP

Megatron

NeMo

AWS Batch

Job description

Amazon's SageMaker AI Training Job team is seeking a Software Development Engineer to help build and operate mission-critical systems at scale, enabling training jobs across hundreds of thousands of machines with minimal failure.

You will own components end-to-end, integrate containerized training frameworks, contribute to SDKs and CDK, and collaborate with ML scientists to translate requirements into self-service platform capabilities. This is a hands-on, high-impact role in Bellevue, WA.