Senior Data Infra Engineer — Spark on Kubernetes

Showcify, Inc.; San Mateo (CA); On-site; USD 243,000 - 295,000; Full time

Benefits offered by this job

Equity compensation

Job summary

Roblox is seeking a Senior Software Engineer to own and operate the Spark-on-Kubernetes compute platform, tackling complex distributed systems challenges. You will design APIs, manage schedules, and drive cost-effective, scalable workloads in a multi-tenant environment.

You will lead efforts across data engineering, ML, and analytics teams, shaping the next generation of the Roblox data platform while ensuring reliability and performance. The role is based in San Mateo, CA.

Qualifications

• 5+ years of experience building and operating production-grade distributed systems or infrastructure platforms.
• Hands-on experience running Apache Spark on Kubernetes (EKS, GKE, or self-managed).
• Strong experience operating production Kubernetes clusters at scale, including upgrades, node lifecycle, autoscaling, and operators.
• Strong proficiency in Go, Java, Scala, or Clojure with a track record of backend code.
• Experience with AWS or GCP and infrastructure-as-code.
• Operational ownership including on-call, incident response, and observability.
• Technical leadership to influence design decisions and mentor engineers.

Responsibilities

• Build and own core compute platform services: design, build, and operate Spark job submission APIs and libraries.
• Launch interactive Spark at scale: deliver Spark Connect for notebook workflows and manage EKS-backed compute.
• Run a safe, automated cluster fleet: build fleet management automation and blue-green rollouts.
• Make jobs run better: develop an auto-tuner using Spark history and event logs.
• Own batch scheduling on Kubernetes: operate and upgrade YuniKorn-based scheduling.
• Shape what we adopt next: evaluate native Spark acceleration and benchmark Roblox workloads.
• Be an owner of reliability and cost: define SLOs, observability, and reduce toil and spend.
• Partner across the company: collaborate with data engineering, ML, analytics, and infra teams.

Skills

Spark on Kubernetes

Kubernetes depth

Go/Java/Scala/Clojure

Cloud fluency

Operational ownership

Technical leadership

Education

B.S. in Computer Science or equivalent

Tools

Spark Operator

Karpenter

Helm

Terraform

Argo

YuniKorn/Volcano/Kueue

Job description

Roblox is seeking a Senior Software Engineer to own and operate the Spark-on-Kubernetes compute platform, tackling complex distributed systems challenges. You will design APIs, manage schedules, and drive cost-effective, scalable workloads in a multi-tenant environment.

You will lead efforts across data engineering, ML, and analytics teams, shaping the next generation of the Roblox data platform while ensuring reliability and performance. The role is based in San Mateo, CA.