Principal Data Orchestration Engineer

Roblox Corporation; San Mateo (CA); In loco; USD 295.000 - 345.000; Tempo pieno

Descrizione del lavoro

Roblox Corporation is seeking a Principal Software Engineer for Data Platform Orchestration in the San Mateo area. You will own multi-year architecture for Airflow, lead Kubernetes-based orchestration for Spark workloads, and build tooling and observability for internal data teams.

You will drive platform reliability, mentor engineers, and partner with data governance, science, and product teams to align roadmaps and deliver scalable data workflows.

Competenze

• B.S. equivalent in CS or sufficient experience.
• 10+ years of experience designing, building, and operating production-grade, large-scale distributed systems.
• Deep, hands-on experience designing, running, and scaling a production data-related workflow orchestrator or job scheduler (thousands of workflows or DAGs a day). Apache Airflow is ideal, but experience with an equivalent system is welcome.
• Proven experience managing and scaling production Kubernetes clusters on AWS (EKS), including Helm, autoscaling, and GitOps-style deployment.
• Strong proficiency in Python, plus Go or Java, with a record of writing clean, well-tested platform and tooling code.
• Working knowledge of Spark jobs on Kubernetes and how orchestration interacts with them.
• Experience building platforms used by other engineers, with APIs, observability, reliability, and developer experience.
• A track record of influencing direction across a large engineering organization and driving multi-quarter projects from concept to production.
• Extreme Ownership: define problems, unblock dependencies, and see complex work through to production.

Mansioni

• Define the Orchestration Strategy: Own multi-year architecture for Airflow and platform evolution.
• Scale and Harden Airflow: remove scaling limits across scheduler, metadata, executors, and DAG parsing.
• Own Orchestration on Kubernetes: lead design and operation of EKS clusters for orchestration and Spark workloads.
• Build Platform Tooling for Internal Engineers: design APIs, operators, providers, and self-service tooling.
• Drive Observability and Reliability: establish SLOs, alerts, and diagnostics; lead incident reviews and root-cause fixes.
• Lead Across Teams: align roadmaps with data platform, governance, data science, and product engineering.
• Raise the Engineering Bar: set standards for code quality and operational readiness; mentor engineers.

Conoscenze

Python

Go or Java

Kubernetes

Apache Airflow

Spark on Kubernetes

APIs

Observability

Platform engineering

Terraform

OpenTelemetry

Formazione

B.S. in CS

Strumenti

Airflow

Dagster/Prefect/Temporal

Kubernetes (EKS)

Helm

Karpenter

Terraform

Prometheus/OpenTelemetry

Descrizione del lavoro

Roblox Corporation is seeking a Principal Software Engineer for Data Platform Orchestration in the San Mateo area. You will own multi-year architecture for Airflow, lead Kubernetes-based orchestration for Spark workloads, and build tooling and observability for internal data teams.

You will drive platform reliability, mentor engineers, and partner with data governance, science, and product teams to align roadmaps and deliver scalable data workflows.