ML Infra Engineer for Petabyte-Scale Training (SF On-Site)

David Joseph & Company; San Francisco (CA); On-site; USD 200,000 - 400,000; Full time

Job summary

David Joseph & Company seeks a talented ML infrastructure engineer to own the distributed training and inference backbone for a large foundation model. You will stand up clusters, build data pipelines for petabyte-scale datasets, and squeeze GPU performance across model scales.

You will work with FSDP/DeepSpeed, NVIDIA GPUs, Linux, Python, C++, Kubernetes/Docker, and major cloud platforms (GCP/AWS/Azure). Relocation supported for San Francisco, on-site five days weekly.

Qualifications

• 2–10 years building ML infra for foundation models.
• Experience with large-scale distributed training from scratch.
• Background in science-focused or physical-AI domains is a plus.

Responsibilities

• Design, deploy, and maintain large distributed ML training and inference clusters.
• Build end-to-end pipelines to manage petabyte-scale datasets and training.
• Research training approaches and parallelization techniques.
• Profile and optimize low-level GPU operations for performance.
• Track research developments and apply new ideas.

Skills

Distributed training

Large-scale infra

Linux

Python

C++

Kubernetes

Docker

Cloud platforms

Tools

FSDP

DeepSpeed

CUDA

NVIDIA GPUs

Job description

David Joseph & Company seeks a talented ML infrastructure engineer to own the distributed training and inference backbone for a large foundation model. You will stand up clusters, build data pipelines for petabyte-scale datasets, and squeeze GPU performance across model scales.

You will work with FSDP/DeepSpeed, NVIDIA GPUs, Linux, Python, C++, Kubernetes/Docker, and major cloud platforms (GCP/AWS/Azure). Relocation supported for San Francisco, on-site five days weekly.