Senior Staff Software Engineer, ML Training Infrastructure

stackav

United States

Remote

USD 180,000 - 240,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

stackav is seeking a senior leader to provide technical direction and hands-on engineering for a high-performance, multi-tenant ML training platform. You will collaborate across infrastructure, ML, autonomy, and safety teams to accelerate the full model workflow—from data processing and dataset preparation to training, validation and deployment.

We value expertise in distributed training, scalable data pipelines, and building reliable infrastructure for multiple modeling teams.

Qualifications

  • Eight or more years of experience with machine learning platforms and applications.
  • Strong Python skills and experience with PyTorch, Ray, Spark, or equivalent technologies.
  • Track record of building scalable, reliable infrastructure while supporting multiple modeling teams.
  • Ability to explain technical trade-offs and align partners across functions, with strong analytical and communication skills.

Responsibilities

  • Design and evolve platform components for orchestration, scheduling, training abstractions, control plane, observability, and performance tuning.
  • Build end-to-end model pipelines covering log processing, feature extraction, dataset schemas and storage, model configuration, training, and profiling.
  • Measure infrastructure performance and diagnose bottlenecks across the stack.
  • Improve distributed training throughput using topology-aware software and hardware approaches.
  • Develop abstractions and tools that help ML engineers iterate independently and efficiently.
  • Promote effective use of open-source profiling and optimization tools, and maintain strong engineering standards.

Skills

Python
Distributed training
Data pipelines
Communication
Problem solving
Analytical thinking
Stakeholder alignment

Education

Bachelor’s or Master’s degree in computer science/engineering or related field

Tools

PyTorch
Ray
Spark

Job description

Role overview

Provide technical direction and hands-on engineering for a high-performance, multi-tenant machine learning training platform. The platform supports the full model workflow, from processing data and preparing datasets through training, validation, profiling, and deployment. You will work across infrastructure, machine learning, autonomy, and safety teams to improve development speed and reliability.

Responsibilities
  • Design and evolve platform components for orchestration, scheduling, training abstractions, control plane, observability, and performance tuning.
  • Build end-to-end model pipelines covering log processing, feature extraction, dataset schemas and storage, model configuration, training, and profiling.
  • Measure infrastructure performance and diagnose bottlenecks across the stack.
  • Improve distributed training throughput using topology-aware software and hardware approaches.
  • Develop abstractions and tools that help machine learning engineers iterate independently and efficiently.
  • Promote effective use of open-source profiling and optimization tools, and maintain strong engineering standards.
Requirements
  • Bachelor’s or master’s degree in computer science, engineering, or a related field.
  • Eight or more years of experience with machine learning platforms and machine learning applications.
  • Demonstrated success optimizing training with distributed methods and building production data pipelines and loaders for large workloads.
  • Strong Python skills and experience with frameworks or tools such as PyTorch, Ray, Spark, or equivalent technologies.
  • Track record of building scalable, reliable infrastructure while supporting multiple modeling teams.
  • Ability to explain technical trade-offs and align partners across functions, alongside strong analytical, problem-solving, and communication skills.
Nice to have
  • Experience with autonomous vehicles or model development.
  • Familiarity with high-speed interconnects and distributed training hardware is relevant to the platform’s performance goals.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Staff Software Engineer, ML Training Infrastructure
Staff Software Engineer, ML Training Infrastructure

stackav • United States

Remote
USD 180,000 - 240,000
Software Engineer – ML Infrastructure
Software Engineer – ML Infrastructure

People Culture Talent • San Francisco (CA)

On-site
USD 190,000 - 230,000
Senior Machine Learning Engineer, Platform
Senior Machine Learning Engineer, Platform

block • United States

Remote
USD 140,000 - 210,000
Remote work
Flexible time off
Medical coverage
+2
Member of Technical Staff
Member of Technical Staff

Harrison Clarke • San Francisco (CA)

On-site
USD 180,000 - 280,000
Machine Learning Infrastructure Engineer
Machine Learning Infrastructure Engineer

Alexander Chapman • New York (NY)

On-site
USD 140,000 - 200,000
Competitive salary
Equity
Health, dental, and vision insurance
Senior Software Engineer - ML Infrastructure
Senior Software Engineer - ML Infrastructure

FDE Times • Sunnyvale (CA)

On-site
USD 150,000 - 210,000
Equity
Health Insurance
Dental Insurance
+7
Senior Software Engineer, Machine Learning Infrastructure & Automation
Senior Software Engineer, Machine Learning Infrastructure & Automation

fal-ai • United States

Remote
USD 150,000 - 210,000
Health insurance
Dental insurance
Vision insurance
Senior Machine Learning Engineer (AI Infrastructure)
Senior Machine Learning Engineer (AI Infrastructure)

Robinhood • Menlo Park (CA)

On-site
USD 200,000 - 320,000
Sr Software Engineer - ML Infrastructure
Sr Software Engineer - ML Infrastructure

Uber • Sunnyvale (CA)

On-site
USD 180,000 - 260,000
Bonus Program
Equity Awards
401(k) Plan
+1
Senior Staff ML Training Platform Engineer
Senior Staff ML Training Platform Engineer

Stack AV • United States

Remote
USD 180,000 - 240,000