Senior ML Data Infrastructure Architect

Unity; Mountain View (CA); On-site; USD 200,000 - 261,000; Full time

Benefits offered by this job

Comprehensive health insurance

Employee stock ownership

Commute subsidy

Generous vacation and personal days

Retirement/pension plans

Job summary

Unity seeks a Senior Machine Learning Engineer, Data Infrastructure, to design and evolve a large-scale offline data platform supporting batch and streaming workloads. You will own data pipelines, training dataset generation, and orchestration with Flyte/Airflow, ensuring reliability, observability, and cost efficiency across distributed systems.

You will partner with ML engineers to enable efficient model iteration, build reproducible experiment workflows, and lead architectural improvements

Qualifications

• Experience with distributed computing frameworks for data processing.
• Experience building infrastructure for training data generation or ML feature pipelines.
• Experience optimizing big data pipelines for cost efficiency.
• Strong Python programming skills.
• Experience with data lakes, warehouses, orchestration systems, streaming platforms.
• Strong systems thinking for performance, scalability, reliability, and cost tradeoffs.
• Ability to lead technical direction across teams.

Responsibilities

• Develop infrastructure for batch and streaming big data processing using Flink, Spark, Ray.
• Design large-scale data pipelines to generate training datasets for ML.
• Integrate pipelines with workflow orchestration systems for multi-stage training workflows.
• Improve reproducibility and observability via dataset validation and automated testing.
• Optimize performance and resource use across distributed compute systems.
• Collaborate with ML engineers to enable large-scale experimentation and model iteration.
• Lead architectural improvements to keep offline data pipelines scalable and cost-efficient.

Skills

Flink

Spark

Ray

Python

Distributed processing

Data infrastructure

Airflow

Flyte

Observability

System design

Tools

Flyte

Airflow

Ray

Job description

Unity seeks a Senior Machine Learning Engineer, Data Infrastructure, to design and evolve a large-scale offline data platform supporting batch and streaming workloads. You will own data pipelines, training dataset generation, and orchestration with Flyte/Airflow, ensuring reliability, observability, and cost efficiency across distributed systems.

You will partner with ML engineers to enable efficient model iteration, build reproducible experiment workflows, and lead architectural improvements