Staff ML Infrastructure Engineer - Distributed Training

Jobtailor; California (MO); On-site; USD 150,000 - 210,000; Full time

Job summary

Jobtailor seeks an experienced AI/ML infrastructure engineer to design, implement, and scale training workflows on AWS using Kubernetes and Python. You will optimize distributed training, improve model serving, and enhance resource utilization for large-scale models.

Collaborate with data scientists to streamline pipelines, contribute to AutoML tools, and advance orchestration. Ideal candidates hold a PhD or MS in CS with 5+ years of hands-on experience and strong communication and teamwork.

Qualifications

• PhD or Master’s in computer science or related field and 5+ years of hands-on industry experience.
• Proven proficiency with Python and developing systems, frameworks and SDKs.
• Experience with infrastructure and understanding of model serving, training, orchestration, and management of GPU resources.
• Experience with machine learning and distributed PyTorch.
• Strong critical thinking, analytical and quantitative problem-solving ability.
• Excellent communication, relationship skills and a strong teammate.
• Experience with KubeFlow, MLFlow, Ray, SageMaker, or similar (added plus).
• Experience with PyTorch distributed, MPI, Megatron, Horovod and other AI training frameworks (added plus).

Responsibilities

• Design, develop, and maintain robust AI/ML infrastructure solutions.
• Improve distributed training frameworks and scalability on GPUs.
• Enhance resiliency, elasticity, data loading, and model parallelism support.
• Improve orchestration and scheduling for faster model training.
• Scale jobs and enable faster experimentation with AutoML tools.
• Collaborate with data scientists to streamline training pipelines and optimize resource utilization.
• Drive innovation in infrastructure practices supporting ML research and development.

Skills

Python Proficiency

Kubernetes Experience

Distributed PyTorch Knowledge

AI/ML Infrastructure Development

GPU Resource Management

Education

PhD in Computer Science

Master’s in Computer Science

Tools

KubeFlow

MLFlow

Ray

SageMaker

PyTorch Distributed

MPI

Megatron

Horovod

Job description

Jobtailor seeks an experienced AI/ML infrastructure engineer to design, implement, and scale training workflows on AWS using Kubernetes and Python. You will optimize distributed training, improve model serving, and enhance resource utilization for large-scale models.

Collaborate with data scientists to streamline pipelines, contribute to AutoML tools, and advance orchestration. Ideal candidates hold a PhD or MS in CS with 5+ years of hands-on experience and strong communication and teamwork.