Senior ML Infrastructure Engineer - Scalable Pipelines

Unity Technologies; United States; Remote; USD 180,000 - 240,000; Full time

Job summary

Unity Technologies is seeking a senior ML engineer to design and evolve a large-scale offline platform for training data generation, dataset preparation, and distributed model training. You will build reliable infrastructure, enhance reproducibility, and lead architectural improvements across distributed systems.

Collaborating with ML engineers and platform teams, you will shape training datasets, manage model training, validation, and delivery to distributed training systems while ensuring

Qualifications

• Proven experience building large-scale ML pipelines.
• Experience with distributed computing frameworks (Ray, Spark, Flink) and Ray ecosystem.
• Proficiency in Python and large-scale distributed workloads.
• Experience with data infrastructure and orchestration systems.

Responsibilities

• Design and operate large-scale data pipelines that generate training datasets used for machine learning training and experimentation
• Develop infrastructure that supports distributed training workflows using technologies such as PyTorch, Ray Data, and Ray Train, etc.
• Integrate ML pipelines with workflow orchestration systems (e.g., Flyte, Airflow, or similar) to enable reliable multi-stage training workflows
• Improve reproducibility and observability of ML pipelines through dataset validation, monitoring, and automated testing
• Optimize performance and resource utilization across distributed compute systems used for data processing and model training
• Partner closely with ML engineers to enable efficient large-scale experimentation and model iteration
• Lead architectural improvements to ensure our offline ML pipelines remain scalable, reliable, and cost-efficient

Skills

Large-scale ML pipelines

Distributed computing

Python

System design

Tools

Ray Data

Ray Train

Flyte

Airflow

Job description

Unity Technologies is seeking a senior ML engineer to design and evolve a large-scale offline platform for training data generation, dataset preparation, and distributed model training. You will build reliable infrastructure, enhance reproducibility, and lead architectural improvements across distributed systems.

Collaborating with ML engineers and platform teams, you will shape training datasets, manage model training, validation, and delivery to distributed training systems while ensuring