Principal Software Engineer, Orchestration

Game Companies

San Mateo (CA)

On-site

USD 295,000 - 345,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Roblox is seeking a Principal Software Engineer for the Data Platform Orchestration team in San Mateo, CA. You will define the multi-year Airflow-based orchestration strategy and own Kubernetes-based execution for data workloads, collaborating with data science, governance, and product engineering to shape a scalable platform.

You will lead design decisions, drive reliability, and mentor engineers while delivering observable, high-performance systems at scale.

Qualifications

  • B.S. in CS or equivalent experience.
  • 10+ years designing, building, and operating production-scale distributed systems.
  • Deep, hands-on workflow orchestration experience with Airflow or equivalent (Dagster/Prefect/Temporal).
  • Strong Kubernetes on AWS (EKS) with Helm and autoscaling knowledge.
  • Proficient in Python and Go/Java with production-grade tooling experience.

Responsibilities

  • Define the multi-year orchestration strategy for Airflow and the surrounding platform.
  • Scale and Harden Airflow across scheduler, metadata store, executors, and DAGs.
  • Own Kubernetes orchestration on EKS for workflows and Spark workloads.
  • Build APIs, operators, libraries, and self-service tooling for data teams.
  • Drive observability, SLOs, monitoring, and incident reviews.
  • Lead cross-team alignment and governance across engineering groups.
  • Raise engineering standards for code quality and reliability.

Skills

Airflow
Kubernetes
Python
Go/Java
Distributed systems
Platform APIs
Observability
Leadership
Terraform/IaC
Spark on Kubernetes

Education

B.S. in CS or equivalent

Tools

Apache Airflow
Kubernetes (EKS)
Docker
Helm
Spark on Kubernetes
Dagster/Prefect/Temporal
Terraform
Prometheus/OpenTelemetry

Job description

Technical Engineering

5+ years

Backend Development

Every day, tens of millions of people come to Roblox to explore, create, play, learn, and connect with friends in 3D immersive digital experiences– all created by our global community of developers and creators.

At Roblox, we’re building the tools and platform that empower our community to bring any experience that they can imagine to life. Our vision is to reimagine the way people come together, from anywhere in the world, and on any device.We’re on a mission to connect a billion people with optimism and civility, and looking for amazing talent to help us get there.

A career at Roblox means you’ll be working to shape the future of human interaction, solving unique technical challenges at scale, and helping to create safer, more civil shared experiences for everyone.

Roblox’s data platform powers analytics, ML, and product decisions for a platform serving 200M+ daily active users. As a Principal Software Engineer in the Data Platform Orchestration team, you will be the technical leader for the system that schedules and runs the company’s data workloads. Our team owns Apache Airflow and the Kubernetes (EKS) infrastructure underneath it, running 8,000+ DAGs every day, most of them Spark jobs. You will set the long‑term architecture for how Roblox orchestrates data at scale, and you will be hands‑on in making it faster, more reliable, and easier for hundreds of engineers to use.

You Will
  • Define the Orchestration Strategy: Own the multi-year architecture for Airflow and the surrounding platform, including scheduler and executor design, multi‑tenancy, upgrades and migrations, and where the platform should evolve next.
  • Scale and Harden Airflow: Find and remove scaling limits across the scheduler, metadata database, executors, and DAG parsing. Make sure thousands of daily DAGs run with predictable latency and strong SLAs.
  • Own Orchestration on Kubernetes: Lead the design and operation of our EKS clusters for orchestration and Spark workloads, including autoscaling, resource isolation, cost efficiency, and deployment automation.
  • Build Platform Tooling for Internal Engineers: Design the APIs, custom operators and providers, libraries, and self‑service tooling that let data teams build, deploy, and monitor pipelines safely.
  • Drive Observability and Reliability: Establish SLOs, alerting, and diagnostics for pipelines and platform components. Lead incident reviews and fix the root causes in code, not just in runbooks.
  • Lead Across Teams: Work with data platform, governance, data science, and product engineering teams to align roadmaps, guide technical decisions, and build consensus on complex initiatives.
  • Raise the Engineering Bar: Set standards for code quality, design, and operational readiness. Mentor staff, senior, and mid‑level engineers.
You Have
  • B.S. equivalent in CS or sufficient experience.
  • [10+] years of experience designing, building, and operating production‑grade, large‑scale distributed systems.
  • Workflow Orchestration Depth: Deep, hands‑on experience designing, running, and scaling a production data‑related workflow orchestrator or job scheduler (thousands of workflows or DAGs a day). Apache Airflow is ideal, but experience with an equivalent system (Dagster, Prefect, Temporal, or a large in‑house scheduler) is also welcome. You understand scheduler and executor design, metadata store performance, task isolation, retries and idempotency, and safe upgrades and migrations.
  • Kubernetes Expertise: Proven experience managing and scaling production Kubernetes clusters on AWS (EKS), including Helm, autoscaling (Karpenter or Cluster Autoscaler), and GitOps‑style deployment.
  • Software Engineering Foundations: Strong proficiency in Python, plus Go or Java, with a record of writing clean, well‑tested platform and tooling code. You are a software engineer who builds and improves systems, not someone who mainly handles tickets and operations.
  • Data Workload Familiarity: Working knowledge of how Spark jobs run on Kubernetes (Spark on K8s, spark‑operator, or EMR on EKS) and how orchestration interacts with them.
  • Platform Mindset: Experience building platforms used by other engineers, with strong instincts for APIs, observability, reliability, and developer experience.
  • Cross‑Organizational Technical Leadership: A track record of influencing direction across a large engineering organization and driving multi‑quarter projects from concept to production.
  • Extreme Ownership: You define the problem when requirements are unclear, unblock dependencies, and see complex work through to stable production.
  • Bonus Points:
    • Contributions to Apache Airflow or related open‑source projects (Airflow providers, Helm chart, spark‑operator, Karpenter, Argo CD).
    • Experience with other orchestration systems (Dagster, Temporal, Argo Workflows, Flyte) and the judgment to know when each fits.
    • Experience with Terraform or other infrastructure‑as‑code, Prometheus/OpenTelemetry, and cost optimization or chargeback for shared platforms.
    • Experience operating infrastructure at consumer‑internet scale.

For roles that are based at our headquarters in San Mateo, CA: The starting base pay for this position is as shown below. The actual base pay is dependent upon a variety of job‑related factors such as professional background, training, work experience, location, business needs and market demand. Therefore, in some circumstances, the actual salary could fall outside of this expected range. This pay range is subject to change and may be modified in the future. All full‑time employees are also eligible for equity compensation and for benefits as described on this page.

Annual Salary Range$295,250—$345,040 USD

Roles that are based in an office are onsite Tuesday, Wednesday, and Thursday, with optional presence on Monday and Friday (unless otherwise noted).

Roblox provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state or local laws. Roblox also provides reasonable accommodations to candidates with qualifying disabilities or religious beliefs during the recruiting process.

For US based roles only, please note the Company may not be able to employ candidates for this role who have United States work authorization related to certain U.S. visa categories, or support future H‑1B sponsorship at this time.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Principal Software Engineer, Orchestration
Principal Software Engineer, Orchestration

Roblox Corporation • San Mateo (CA)

On-site
USD 295,000 - 345,000
Equity compensation
Principal Software Engineer, Orchestration
Principal Software Engineer, Orchestration

Showcify, Inc. • San Mateo (CA)

On-site
USD 295,000 - 345,000
Principal Software Engineer (Orchestration)
Principal Software Engineer (Orchestration)

Roblox Corporation • San Mateo (CA)

On-site
USD 295,000 - 345,000
Principal Software Engineer, Data Tooling
Principal Software Engineer, Data Tooling

Roblox Corporation • San Mateo (CA)

On-site
USD 295,000 - 345,000
Equity compensation
Senior Software Engineer, Data Infrastructure Compute
Senior Software Engineer, Data Infrastructure Compute

Showcify, Inc. • San Mateo (CA)

On-site
USD 243,000 - 295,000
Equity compensation
Principal Software Engineer, Creator Analytics
Principal Software Engineer, Creator Analytics

Roblox Corporation • San Mateo (CA)

On-site
USD 345,000 - 399,000
Equity compensation
Benefits described on page
Senior Software Engineer, Data Infrastructure Compute
Senior Software Engineer, Data Infrastructure Compute

Roblox Corporation • San Mateo (CA)

On-site
USD 243,000 - 295,000
Senior Software Engineer, Realtime Analytics Platform
Senior Software Engineer, Realtime Analytics Platform

Roblox Corporation • San Mateo (CA)

On-site
USD 243,000 - 295,000
Equity compensation
Benefits described on Roblox page
Principal Software Engineer, Creator Analytics
Principal Software Engineer, Creator Analytics

Speedrun Talent Network • San Mateo (CA)

On-site
USD 345,000 - 399,000
Senior Software Engineer, Realtime Analytics Platform
Senior Software Engineer, Realtime Analytics Platform

Game Companies • San Mateo (CA)

On-site
USD 243,000 - 295,000
Equity compensation
Benefits described on page