Senior Systems Software Engineer - GPU-Accelerated Spark

NVIDIA Corporation; Santa Clara (CA); On-site; USD 184,000 - 288,000; Full time

Benefits offered by this job

Equity

Benefits package

Job summary

NVIDIA is seeking a Senior Systems Software Engineer for the Apache Spark Acceleration group in Santa Clara, CA. You will design, build, and operate a hosted Spark service on Kubernetes, extending the cuDF Spark runtime and workload router in Java/Scala, with exposure to Delta Lake and Unity Catalog integrations.

You will own one or more services end to end, contribute to production stability, and improve deployment pipelines.

Qualifications

• BS, MS, or PhD in Computer Science, Computer Engineering, or a closely related field (or equivalent experience)
• 9+ years of experience in software development
• Strong distributed-systems fundamentals and proficiency in Java or Scala, plus C++ or Python
• Hands-on experience operating services on Kubernetes, on-premise or cloud
• Experience owning or reviewing a long-lived server: REST front ends, proxies or request routers, JDBC or Thrift services
• Experience supporting enterprise or internal production customers, including on-call
• Experience using AI tools in software development

Responsibilities

• Design, build and operate hosted Spark service control plane on Kubernetes: the Java REST API, the Spark Connect runner, Kyuubi, and deployment using Helm charts and Flux
• Own code review for one or more services end to end and keep them releasable on the release train
• Extend the cuDF Spark runtime and the CPU/GPU workload router in Scala and Java, including interoperability with Delta Lake, Apache Iceberg and Unity Catalog
• Investigate production failures on clusters running more than a thousand concurrent jobs together with the customer team and upstream library teams, and drive fixes to closure
• Instrument metrics, dashboards and alerts; take part in the on-call rotation and keep the runbooks current
• Provide recommendations and feedback to teams regarding decisions surrounding topics such as infrastructure, continuous integration and testing strategy

Skills

Distributed systems

Java

Scala

C++

Python

Kubernetes

REST

On-call support

Education

BS/MS/PhD in Computer Science or related field

Tools

Spark Connect

Kyuubi

Helm

Flux

Delta Lake

Apache Iceberg

Unity Catalog

Job description

NVIDIA is seeking a Senior Systems Software Engineer for the Apache Spark Acceleration group in Santa Clara, CA. You will design, build, and operate a hosted Spark service on Kubernetes, extending the cuDF Spark runtime and workload router in Java/Scala, with exposure to Delta Lake and Unity Catalog integrations.

You will own one or more services end to end, contribute to production stability, and improve deployment pipelines.