ML Performance Engineer – Multi-Node Training & Kernels

Veeda AI; California; On-site; USD 180,000 - 240,000; Full time

Job summary

Veeda AI in California is building the next generation of multimodal foundation world models for Physical AI. You will design and optimize distributed training and inference pipelines on large GPU clusters, focusing on low-latency, high-throughput workloads.

The role requires deep expertise in PyTorch, large-scale parallelism, and CUDA kernel work, with a track record of improving hardware utilization and stability at scale.

Qualifications

• Bachelor's degree in Computer Science, Computer Engineering, or related field.
• Deep hands-on experience with PyTorch and large-scale parallelism stacks on real multi-node jobs.
• Fluency in Python and C++/CUDA with kernel profiling intuition.
• Experience profiling live training/inference with Nsight Systems or PyTorch profiler and translating traces into improvements.
• Expertise in low-precision numerics, kernel authoring, or large-run fault diagnosis.

Responsibilities

• Distributed Training Throughput: own step time and model FLOPs for multi-node video world model training.
• Inference Throughput Optimization: profile latency, GPU utilization, memory pressure, and kernel efficiency; improve throughput via profiling and CUDA tools.
• Precision & Numerical Stability: work with BF16, FP8, and related recipes to ensure stability in video tokenizer and VAE layers.
• Kernels & Compilation: write/tune CUDA and Triton kernels; drive FlashAttention-4 and related integration to reduce step time.
• Communication & Overlap: optimize NCCL collectives and compute/communication overlap across NVLink.
• Fault Diagnostics & Recovery: build detection for silent data corruption and asynchronous checkpointing.

Skills

PyTorch

Python

C++/CUDA

FSDP2

Megatron-Core

DeepSpeed

NCCL

Nsight Systems

Profiling

Education

Bachelor's degree in CS/CE

Tools

NCCL

Nsight Systems

CUDA

PyTorch Profiler

Job description

Veeda AI in California is building the next generation of multimodal foundation world models for Physical AI. You will design and optimize distributed training and inference pipelines on large GPU clusters, focusing on low-latency, high-throughput workloads.

The role requires deep expertise in PyTorch, large-scale parallelism, and CUDA kernel work, with a track record of improving hardware utilization and stability at scale.