Senior ML Infrastructure Engineer: Model Training

Baseten; United States; Remote; USD 200,000 - 275,000; Full time

Benefits offered by this job

Competitive compensation package

Learning and growth opportunities

Inclusive culture

Job summary

Baseten is hiring a Senior Software Engineer - Model Training to build the infrastructure powering large-scale training and fine-tuning of foundation models. You will design distributed training systems, optimize GPU usage, and create scalable pipelines for efficient model adaptation on Baseten’s platform.

The role is highly technical, hands-on, and requires collaboration with product and infra teams to meet real-world customer needs while advancing training technology and tooling.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or related field, or equivalent experience.
• 4+ years of experience in software engineering with a focus on ML infrastructure, distributed systems, or ML platform engineering.
• Hands-on expertise in distributed training frameworks (FSDP, DDP, ZeRO, or similar) and ML frameworks (PyTorch, Transformers, Lightning, TRL).
• Strong understanding of GPU/accelerator performance optimization and scaling techniques.
• Experience designing and operating large-scale systems in production (cloud-native preferred).
• Excellent problem-solving and communication skills, with the ability to work across infrastructure and ML boundaries.

Responsibilities

• Design, build, and maintain distributed training infrastructure for large-scale foundation models
• Implement scalable pipelines for fine-tuning and training across heterogeneous GPU/accelerator clusters
• Optimize training performance through techniques like FSDP, DDP, ZeRO, and mixed precision training
• Contribute to frameworks and tooling that make training workflows efficient, reproducible, and developer-friendly
• Collaborate with cross-functional teams (Product, Forward Deployed Engineering, Inference Infra) to ensure training systems meet real-world requirements
• Research and apply emerging techniques in training efficiency and model adaptation, and productionize them in the Baseten platform
• Participate in code reviews, system design discussions, and technical deep dives to maintain a high engineering bar

Skills

Distributed systems

ML infrastructure

PyTorch

GPU optimization

Education

Bachelor's degree in CS or related field

Tools

FSDP

DDP

ZeRO

Transformers

Job description

Baseten is hiring a Senior Software Engineer - Model Training to build the infrastructure powering large-scale training and fine-tuning of foundation models. You will design distributed training systems, optimize GPU usage, and create scalable pipelines for efficient model adaptation on Baseten’s platform.

The role is highly technical, hands-on, and requires collaboration with product and infra teams to meet real-world customer needs while advancing training technology and tooling.