Senior Compiler Engineer — GPU DL Training, Equity

NVIDIA Corporation; Redmond; On-site; AUD 264,000 - 413,000; Full time

Benefits offered by this job

Equity

Benefits

Job summary

NVIDIA Corporation is seeking a senior engineer to design and optimize compilers and runtimes for large-scale deep learning training on multi-GPU systems. You will collaborate with JAX/XLA teams and contribute to the open-source JaxPP project, helping accelerate next-gen DL software.

The role requires deep expertise in compilers, distributed systems, and DL frameworks, with strong C/C++ and Python skills, and a track record of delivering performance-at-scale across complex workloads.

Qualifications

• Masters or PhD in Computer Science, Computer Engineering, or related field or equivalent experience.

Responsibilities

• Design and implement program transformations on JAX programs (jaxpr) for large-scale training.
• Develop technologies and products involving optimizing compilers and runtime execution frameworks for accelerating deep learning training on large scale distributed systems.
• Work on compiling a graph intermediate representation (IR) to target specific code, and build the distributed runtime that dispatches per-stage programs to many GPUs and moves data between stages through communication libraries such as NCCL.
• Contribute to advancing the state of the art in compiler technology for delivering high performance on large-scale training workloads, including mixture-of-experts models.
• Collaborate with members of the software framework teams (for example JAX and XLA) and the hardware architecture teams to accelerate the next generation of deep learning software, and work on integrating JaxPP into JAX training frameworks, including MaxText.
• Be a core contributor to the open-source JaxPP project: keeping it working with new JAX releases, testing, performance tracking, and working with teams who adopt it for their training runs.

Skills

C/C++ programming

Python programming

Distributed systems

Deep learning training

Communication

Independent work

Education

Master's/PhD in CS/CE or related field

Tools

MPI

NCCL

NVSHMEM

XLA

JAX

PyTorch

CUDA

Job description

NVIDIA Corporation is seeking a senior engineer to design and optimize compilers and runtimes for large-scale deep learning training on multi-GPU systems. You will collaborate with JAX/XLA teams and contribute to the open-source JaxPP project, helping accelerate next-gen DL software.

The role requires deep expertise in compilers, distributed systems, and DL frameworks, with strong C/C++ and Python skills, and a track record of delivering performance-at-scale across complex workloads.