Senior Data Infrastructure Engineer: Scalable ML Pipelines

ironSource; Mountain View (CA); On-site; USD 200,000 - 261,000; Full time

Benefits offered by this job

Health insurance

Stock options

Retirement plans

Generous vacation

Job summary

Unity seeks a Senior Machine Learning Engineer for Data Infrastructure to design and evolve a large-scale offline data platform. You will own infrastructure for training datasets, feature pipelines, and data workflows, partnering with ML engineers and platform teams to handle growing data volumes and complex training workloads.

You will drive reliability, scalability, and cost efficiency across distributed systems, enabling efficient model iteration and experimentation at scale.

Qualifications

• Experience with batch and streaming data pipelines.
• Experience with distributed workloads.
• Strong Python skills.
• Experience with data lakes and orchestration systems.
• Ability to lead architectural decisions.

Responsibilities

• Develop infrastructure for batch and stream big data processing with Flink, Spark, Ray.
• Design and operate large-scale data pipelines for training datasets.
• Integrate pipelines with Flyte, Airflow, or similar orchestration systems.
• Improve reproducibility and observability with validation, monitoring, testing.
• Optimize performance and resource use across distributed compute systems.
• Collaborate with ML engineers to enable large-scale experimentation.
• Lead architectural improvements to keep offline pipelines scalable and cost-efficient.

Skills

Distributed data processing

Python programming

Data pipelines

ML data generation

Cost optimization

Tools

Flink

Spark

Ray

Job description

Unity seeks a Senior Machine Learning Engineer for Data Infrastructure to design and evolve a large-scale offline data platform. You will own infrastructure for training datasets, feature pipelines, and data workflows, partnering with ML engineers and platform teams to handle growing data volumes and complex training workloads.

You will drive reliability, scalability, and cost efficiency across distributed systems, enabling efficient model iteration and experimentation at scale.