Principal Systems Software Engineer, GPU-Accelerated Spark

Nvidia Corporation; Santa Clara (CA); On-site; USD 272,000 - 431,250; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA seeks a Sr. Principal Systems Software Engineer for the Apache Spark Acceleration group to drive GPU-accelerated data processing in production deployments.

You'll develop Java, Scala, and CUDA/C++ libraries to accelerate Spark DataFrames and I/O on common formats, collaborating with OSS communities and distributed systems teams at scale. A strong background in distributed systems and open source ecosystems is required.

Qualifications

• 15+ years of work experience in software development.
• BS, MS, or PhD in Computer Science, Computer Engineering, or closely related field (or equivalent experience).
• Outstanding technical skills in designing and implementing high-quality distributed systems.
• Excellent programming skills in C++, Java, and/or Scala.
• Ability to work with teams across organizational boundaries and geographies.
• Familiarity with the open source data platform ecosystem (Apache Spark, Velox, Presto, Apache Arrow, Apache DataFusion, etc.).
• Highly motivated with strong interpersonal skills.
• Database query optimization is a strong plus.

Responsibilities

• Develop Java, Scala and CUDA/C++ libraries to accelerate DataFrames and I/O operations on Parquet, ORC and JSON.
• Enable interoperability with table formats such as Apache Iceberg and Delta Lake, and metastores such as Unity Catalog.
• Work with open source communities to enhance cuDF, CCCL and UCX through discussion and code contributions.
• Collaborate with distributed systems teams to craft solutions to distributed processing problems at scale.
• Provide recommendations and feedback to teams regarding infrastructure, CI and testing strategy.
• Build, test and optimize CUDA/C++ libraries across different platforms.

Skills

C++

Java

Scala

Distributed systems

Open source

Team collaboration

Problem solving

Education

BS/MS/PhD in CS/CE

Tools

CUDA/C++

Apache Spark

cuDF/UCX

Iceberg/Delta Lake

Job description

NVIDIA seeks a Sr. Principal Systems Software Engineer for the Apache Spark Acceleration group to drive GPU-accelerated data processing in production deployments.

You'll develop Java, Scala, and CUDA/C++ libraries to accelerate Spark DataFrames and I/O on common formats, collaborating with OSS communities and distributed systems teams at scale. A strong background in distributed systems and open source ecosystems is required.