Senior ML Infra Lead: Data Pipelines & Training Reliability

Walden Robotics; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Salary and annual bonus

Company equity

Company-subsidized insurance programs

401(k) with company match

Flexible PTO

Daily lunch

Job summary

Walden Robotics in San Francisco seeks a senior ML Infra engineer to build and scale the data infrastructure that feeds large-scale training. You’ll lead this workstream alongside the data team, driving fast, stable, and reproducible runs for research initiatives.

You’ll mentor engineers, set technical standards, and remove bottlenecks between the data platform and training. The role emphasizes performance measurement and robust reliability in a fast-paced, collaborative environment.

Qualifications

• Experience supporting large-scale ML training infrastructure.
• Deep understanding of distributed-training performance.
• Ability to lead a work stream and make high-leverage infra decisions.
• Rigorous, measurement-driven approach to performance.
• Experience mentoring engineers and setting technical standards.

Responsibilities

• Data Path: Design and scale the data-loading, sharing, and caching that feeds large training runs.
• Training Reliability: Design and implement training-infra efficiency and reliability—throughput, utilization, checkpointing, fault tolerance.
• Bottleneck Removal: Find and remove bottlenecks between the data platform and training.
• Observability: Build tooling that makes training performance and failures easy to understand.
• Technical Leadership: Set standards and mentor MTS engineers across data and training infra.

Skills

Systems / ML Infra

Distributed Training

Technical Judgment

Performance

Leadership

Job description

Walden Robotics in San Francisco seeks a senior ML Infra engineer to build and scale the data infrastructure that feeds large-scale training. You’ll lead this workstream alongside the data team, driving fast, stable, and reproducible runs for research initiatives.

You’ll mentor engineers, set technical standards, and remove bottlenecks between the data platform and training. The role emphasizes performance measurement and robust reliability in a fast-paced, collaborative environment.