Senior AI Training Systems Engineer

Genesis AI; San Carlos (CA); On-site; USD 180,000 - 240,000; Full time

Job summary

Genesis AI seeks a senior ML infrastructure engineer to reduce wall-clock time to convergence by profiling and removing bottlenecks from data pipelines to GPU kernels. You will design and optimize distributed PyTorch training for multi-node GPU clusters, with emphasis on scalability and robustness.

Role requires deep experience in ML infrastructure, production-grade Python, and low-level CUDA/cuDNN/Triton mastery to optimize hardware–software interactions for maximum utilization.

Qualifications

• 8+ years in distributed systems, ML infra, or HPC.
• Production-grade Python expertise.
• Expert CUDA/cuDNN/Triton and CPU–GPU tuning.
• Experience with PyTorch training jobs and parallelism.
• Proven ability to optimize hardware–software for utilization.

Responsibilities

• Profile and remove bottlenecks across training stack.
• Design and optimize distributed PyTorch systems for multi-node GPUs.
• Implement low-level CUDA kernels and integrate with frameworks.
• Optimize CPU/GPU balance, memory, throughput, and networking.
• Develop monitoring and debugging tools for large-scale runs.

Skills

Distributed systems

Python

CUDA optimization

PyTorch training

Kernel optimization

Tools

CUDA

cuDNN

Triton

PyTorch

Job description

Genesis AI seeks a senior ML infrastructure engineer to reduce wall-clock time to convergence by profiling and removing bottlenecks from data pipelines to GPU kernels. You will design and optimize distributed PyTorch training for multi-node GPU clusters, with emphasis on scalability and robustness.

Role requires deep experience in ML infrastructure, production-grade Python, and low-level CUDA/cuDNN/Triton mastery to optimize hardware–software interactions for maximum utilization.