Senior Data Compute Platform Engineer (Spark on Kubernetes)

Roblox; San Mateo (CA); On-site; USD 243,000 - 295,000; Full time

Job summary

Roblox is seeking a Senior Software Engineer for the Data Platform Compute team in San Mateo, CA. You will own and operate the Spark-driven compute platform that powers Roblox analytics, ML, and product decisions, including job submission, interactive notebooks, and cluster lifecycle automation.

You will design scalable compute services, optimize performance, and drive cost efficiency while partnering with data engineering and infrastructure teams to support Roblox’s consumer-scale workloads.

Qualifications

• 5+ years building and operating production-grade distributed systems.
• 0
• Strong experience running Apache Spark on Kubernetes as a platform, including executor/driver behavior, resource tuning, and troubleshooting.
• Proficient in at least one backend language (Go/Java/Scala/Clojure); Python is a plus.
• Cloud fluency with AWS or GCP and infrastructure-as-code practices.
• Experience taking a system from design to production, including on-call and incident response.
• Ability to influence design decisions and mentor engineers across teams.
• Bachelor’s degree in CS or equivalent practical experience.

Responsibilities

• Design, build, and operate core compute platform services behind Spark job submission and APIs.
• Deliver Spark Connect and manage EKS-backed notebook compute with isolation and cost controls.
• Build fleet automation for Kubernetes that supports safe updates and low disruption.
• Develop auto-tuner using Spark history/events to improve performance and reduce costs.
• Own batch scheduling on Kubernetes, including queue design and quotas.
• Evaluate emerging tech (e.g., Comet) and benchmark Roblox workloads to inform choices.
• Define SLOs, improve observability, and reduce toil across a multi-tenant platform.
• Collaborate with data engineering, ML, analytics, and infra teams to migrate users.

Skills

Kubernetes

Apache Spark

Spark on Kubernetes

Distributed systems

Go/Java/Scala/Clojure

Python

Cloud platforms (AWS/GCP)

Operational ownership

Technical leadership

B.S. in CS or equivalent

Education

B.S. in Computer Science or equivalent practical experience

Tools

Terraform

Argo

Karpenter

Helm

Job description

Roblox is seeking a Senior Software Engineer for the Data Platform Compute team in San Mateo, CA. You will own and operate the Spark-driven compute platform that powers Roblox analytics, ML, and product decisions, including job submission, interactive notebooks, and cluster lifecycle automation.

You will design scalable compute services, optimize performance, and drive cost efficiency while partnering with data engineering and infrastructure teams to support Roblox’s consumer-scale workloads.