Senior AI Compiler Engineer — Distributed Training

NVIDIA Corporation

Santa Clara (CA)

On-site

USD 184,000 - 288,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Equity
Benefits

Job summary

NVIDIA in Santa Clara, CA is seeking a world-class Software Engineer with AI compiler and distributed systems experience for its Deep Learning Compiler Technology team.

You will design and implement program transformations on JAX programs, build a distributed runtime across GPUs, and contribute to open-source JaxPP and related tooling while collaborating with JAX/XLA teams and partner companies.

Qualifications

  • Masters or PhD in Computer Science, Computer Engineering, or related field or equivalent experience.
  • 6+ years of relevant work or research experience in compilers, distributed systems, or deep learning training.
  • Compiler design and implementation.
  • Distributed system design, development, and deployment.
  • Experience with deep learning frameworks (JAX, PyTorch).
  • GPU training and inference of deep learning models.
  • C/C++ and Python programming, with excellent software design skills, including debugging, performance analysis, and test design.
  • Experience with parallel computing libraries (MPI). Be able to work independently, define project goals and scope, and lead your own development efforts.
  • Strong communication skills are required, including working directly with engineers at partner companies to debug and tune large training runs, along with the ability to work in a dynamic product-oriented team.

Responsibilities

  • Design and implement program transformations on JAX programs (jaxpr) for large-scale training, on top of unmodified JAX and XLA.
  • Develop technologies and products involving optimizing compilers and runtime execution frameworks for accelerating deep learning training on large scale distributed systems.
  • Work on compiling a graph intermediate representation (IR) to target specific code, and build the distributed runtime that dispatches per-stage programs to many GPUs and moves data between stages through communication libraries such as NCCL.
  • Contribute to advancing the state of the art in compiler technology for delivering high performance on large-scale training workloads, including mixture-of-experts models.
  • Collaborate with members of the software framework teams (for example JAX and XLA) and the hardware architecture teams to accelerate the next generation of deep learning software, and work on integrating JaxPP into JAX training frameworks, including MaxText.
  • Be a core contributor to the open-source JaxPP project: keeping it working with new JAX releases, testing, performance tracking, and working with teams who adopt it for their training runs.

Skills

Compiler design
Distributed systems
Deep learning training
GPGPU programming
MPI
JAX/PyTorch experience

Education

Masters or PhD in Computer Science, Computer Engineering, or related field

Tools

MPI
CUDA
JAX
XLA
PyTorch
NCCL

Job description

NVIDIA in Santa Clara, CA is seeking a world-class Software Engineer with AI compiler and distributed systems experience for its Deep Learning Compiler Technology team.

You will design and implement program transformations on JAX programs, build a distributed runtime across GPUs, and contribute to open-source JaxPP and related tooling while collaborating with JAX/XLA teams and partner companies.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior AI Compiler Engineer
Senior AI Compiler Engineer

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 184,000 - 288,000
Equity
Benefits
Senior AI Compiler Engineer (Distributed DL Training)
Senior AI Compiler Engineer (Distributed DL Training)

NVIDIA Corporation • United States

On-site
USD 184,000 - 288,000
Equity
Benefits
Senior AI Compiler Engineer – Large-Scale DL Training
Senior AI Compiler Engineer – Large-Scale DL Training

NVIDIA Corporation • Redmond (WA)

On-site
USD 184,000 - 288,000
Equity
Benefits
Senior Compiler Engineer - AI
Senior Compiler Engineer - AI

NVIDIA Corporation • United States

On-site
USD 184,000 - 288,000
Equity
Benefits
Senior Compiler Engineer - AI
Senior Compiler Engineer - AI

NVIDIA Corporation • Redmond (WA)

On-site
USD 184,000 - 288,000
Equity
Benefits
Senior Compiler Engineer - AI
Senior Compiler Engineer - AI

2100 NVIDIA USA • Santa Clara (CA)

On-site
USD 184,000 - 288,000
Equity
Benefits
Senior Compiler Engineer - AI
Senior Compiler Engineer - AI

Nvidia Corporation in • Santa Clara (CA)

On-site
USD 184,000 - 288,000
Equity
Benefits
Senior Engineering Manager — JAX Strategy & AI Frameworks
Senior Engineering Manager — JAX Strategy & AI Frameworks

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 272,000 - 431,000
Equity
Benefits
Senior AI Compiler Engineer: Code-Gen & Optimizations
Senior AI Compiler Engineer: Code-Gen & Optimizations

NVIDIA Corporation • Santa Clara (CA)

On-site
USD 152,000 - 242,000
Senior JAX AI Platform Engineer
Senior JAX AI Platform Engineer

NVIDIA Corporation • United States

Remote
USD 180,000 - 320,000