LLM Training Infra Engineer - Distributed Systems

ByteDance; San Jose (CA); On-site; USD 244,800 - 450,000; Full time

Benefits offered by this job

Medical, dental, and vision insurance

401(k) savings plan with company match

Paid parental leave

Short-term and long-term disability coverage

Life insurance

10 paid holidays

10 paid sick days

17 days of Paid Personal Time

Job summary

ByteDance is hiring for a position focused on large-scale LLM training infrastructure in San Jose, California. The successful candidate will conduct research and development on distributed training strategies, optimize performance across hardware, and analyze existing systems for enhancements. The role demands extensive experience with ML systems, programming in Python or C++, and understanding of parallelism strategies. A competitive salary ranging from $244,800 to $450,000 annually, plus benefits, is offered. Opportunities for bonuses and incentives are also available.

Qualifications

• Experience with large-scale distributed training for LLMs.
• Strong programming skills in Python and/or C++.
• Strong background in ML systems / training infrastructure development.
• Proficiency in parallelism strategies (DDP, FSDP).
• Solid understanding of training stack internals (PyTorch, CUDA, NCCL).
• Experience in performance optimization (memory, communication, throughput).

Responsibilities

• Conduct research and development on large-scale LLM training infrastructure.
• Design and optimize distributed training strategies for LLMs.
• Investigate system reliability and resilience techniques.
• Research and optimize network, scheduling, and GPU memory management.
• Analyze performance bottlenecks in exascale training systems.
• Translate research ideas into scalable AI infrastructure solutions.

Skills

Large-scale distributed training for LLMs

Programming skills in Python and/or C++

ML systems / training infrastructure development

Proficiency in parallelism strategies (DDP, FSDP)

Understanding of training stack internals (PyTorch, CUDA, NCCL)

Performance optimization (memory, communication, throughput)

Job description

ByteDance is hiring for a position focused on large-scale LLM training infrastructure in San Jose, California. The successful candidate will conduct research and development on distributed training strategies, optimize performance across hardware, and analyze existing systems for enhancements. The role demands extensive experience with ML systems, programming in Python or C++, and understanding of parallelism strategies. A competitive salary ranging from $244,800 to $450,000 annually, plus benefits, is offered. Opportunities for bonuses and incentives are also available.