Senior ML Data Platform Engineer

LE130 Unity Technologies SF; Mountain View (CA); In loco; USD 210.000 - 273.000; Tempo pieno

Vantaggi offerti da questo lavoro

Health insurance

Commuter subsidy

Stock options

Retirement plans

Vacation & personal days

Parental leave

Snacks & meals

Wellbeing programs

ERGs

Training & development

Volunteering & matching

Descrizione del lavoro

Unity Technologies SF is seeking a Senior Machine Learning Engineer, Data Infrastructure, to design and evolve the offline data platform that enables model training and experimentation at scale. You will build reliable pipelines for training data generation and orchestrate workflows with Flyte or Airflow.

You will partner with ML engineers to improve observability, performance, and cost efficiency across distributed systems, guiding architectural decisions to keep pipelines scalable and robust.

Competenze

• Experience with distributed computing frameworks (Flink, Spark, Ray) for data processing.
• Experience building infrastructure for training data generation or ML feature pipelines.
• Experience optimizing big data pipelines for cost efficiency.
• Strong Python programming skills for large-scale workloads.
• Experience with modern data infrastructure (data lakes, warehouses, orchestration systems, streaming platforms).
• Strong systems thinking for performance, scalability, reliability, and cost tradeoffs.

Mansioni

• Develop infrastructure supporting batch and stream big data processing using Flink, Spark, Ray, etc.
• Design and operate large-scale data pipelines that generate training datasets for ML training and experimentation
• Integrate data pipelines with workflow orchestration systems (Flyte, Airflow) for multi-stage training workflows
• Improve reproducibility and observability of data pipelines via dataset validation, monitoring, and automated testing
• Optimize performance and resource utilization across distributed compute systems
• Partner with ML engineers to enable efficient large-scale experimentation and model iteration
• Lead architectural improvements to keep offline data pipelines scalable, reliable, and cost-efficient

Conoscenze

Distributed computing

Python

Data pipelines

ML data infra

Strumenti

Flink

Spark

Ray

Airflow

Flyte

Descrizione del lavoro

Unity Technologies SF is seeking a Senior Machine Learning Engineer, Data Infrastructure, to design and evolve the offline data platform that enables model training and experimentation at scale. You will build reliable pipelines for training data generation and orchestrate workflows with Flyte or Airflow.

You will partner with ML engineers to improve observability, performance, and cost efficiency across distributed systems, guiding architectural decisions to keep pipelines scalable and robust.