Senior AI Training Infra Engineer - Scale GPU Clusters

Designworks Talent; Bellevue (WA); Hybrid; USD 180,000 - 240,000; Full time

Benefits offered by this job

Medical insurance

401(k) with company match

Paid holidays

Job summary

Designworks Talent is hiring an AI Training Infrastructure Engineer to build and scale the distributed systems powering large-scale AI model training. The role focuses on reliability, efficiency, and operational excellence across GPU clusters to enable researchers and engineers to train and deploy advanced models at scale.

You will collaborate with platform and ML teams, own technically challenging projects, and contribute to the evolution of the AI infrastructure platform in a fast-moving

Qualifications

• Experience with distributed training systems and multi-node GPU infrastructures.
• Experience integrating training systems with production ML pipelines.
• Strong programming skills and ownership of complex projects.

Responsibilities

• Build and scale distributed training infrastructure for large AI models across GPU clusters.
• Design systems to improve training reliability, efficiency, and resource utilization.
• Develop fault tolerance, checkpointing, recovery, and large-scale training workflows.
• Integrate AI models into production training pipelines with platform teams.
• Diagnose and resolve training throughput, stability, and cost issues.
• Create tooling and automation to improve researcher/developer experience.
• Establish best practices for training infra and platform reliability.
• Contribute to evolution of AI infra platform as an early team member.

Skills

Distributed systems

GPU training

Python

Ownership

Tools

Kubernetes

PyTorch Distributed

DeepSpeed

Megatron-LM

Ray

Job description

Designworks Talent is hiring an AI Training Infrastructure Engineer to build and scale the distributed systems powering large-scale AI model training. The role focuses on reliability, efficiency, and operational excellence across GPU clusters to enable researchers and engineers to train and deploy advanced models at scale.

You will collaborate with platform and ML teams, own technically challenging projects, and contribute to the evolution of the AI infrastructure platform in a fast-moving