Lead Scientist – Distributed Foundation Model Training

Institute of Foundation Models; Sunnyvale (CA); On-site; USD 300,000 - 600,000; Full time

Benefits offered by this job

Comprehensive medical, dental, and vision benefits

401K Plan

Generous paid time off

Paid Parental Leave

Employee Assistance Program

Life insurance and disability

Job summary

A leading research institution in Sunnyvale is seeking an experienced AI Engineer to focus on distributed pre-training frameworks and enhance training efficiency. Candidates should have extensive experience in deep learning, particularly in PyTorch or JAX, and strong software engineering skills. The position offers a total compensation of $300K – $600K, alongside comprehensive benefits and opportunities for visa sponsorship.

Qualifications

• 5+ years industry or research experience with large-scale deep learning training.
• Led large-scale transformer pre-training run.
• Experience with distributed training at scale (100+ GPUs).
• Expertise in PyTorch or JAX/Flax and distributed frameworks.
• Strong software engineering skills and ownership of mixed/low-precision paths.

Responsibilities

• Build and scale distributed pre-training frameworks for AI models.
• Implement custom optimizers and experiment infrastructure.
• Optimize performance via kernel fusion and communication tuning.
• Collaborate with global research teams and document designs clearly.

Skills

PyTorch

JAX

DeepSpeed

FSDP

Megatron-LM

CUDA

Triton

NCCL

GLOO

Job description

A leading research institution in Sunnyvale is seeking an experienced AI Engineer to focus on distributed pre-training frameworks and enhance training efficiency. Candidates should have extensive experience in deep learning, particularly in PyTorch or JAX, and strong software engineering skills. The position offers a total compensation of $300K – $600K, alongside comprehensive benefits and opportunities for visa sponsorship.