Production ML Engineer: Large-Scale LLM Training

Anthropic Limited; San Francisco (CA); On-site; USD 350,000 - 850,000; Full time

Benefits offered by this job

Equity donation matching

Generous vacation and parental leave

Flexible working hours

Office in San Francisco

Job summary

Anthropic is seeking a Research Engineer in San Francisco to join our ML Performance and Scaling team. You will help ensure production pretrained models train reliably, efficiently, and at scale, working at the boundary of research and engineering across the full training stack.

The role involves owning pipeline components, debugging across hardware and software, running experiments to improve efficiency, and coordinating incident responses during launches.

Qualifications

• Experience training large language models or deep expertise with JAX, TPU, PyTorch, or large-scale distributed systems.
• Ability to work across research and engineering with a roughly 50/50 split.
• Comfort with on-call production systems and long-hour launches.

Responsibilities

• Own critical aspects of the production pretraining pipeline including model operations and observability.
• Debug and resolve complex issues across the full stack from hardware to training dynamics.
• Design and run experiments to improve training efficiency and model performance.
• Respond to on-call incidents during model launches and coordinate solutions across teams.
• Build and maintain production logging, monitoring dashboards, and evaluation infrastructure.
• Add capabilities to training codebase such as long context support or novel architectures.
• Collaborate with teams in SF and London and with Tokens, Architectures, and Systems teams.

Skills

Large language models training

LLM debugging

Cross-team collaboration

Strong troubleshooting

Communication across time zones

Education

Bachelor's degree

Tools

JAX

TPU

PyTorch

Distributed training frameworks

Job description

Anthropic is seeking a Research Engineer in San Francisco to join our ML Performance and Scaling team. You will help ensure production pretrained models train reliably, efficiently, and at scale, working at the boundary of research and engineering across the full training stack.

The role involves owning pipeline components, debugging across hardware and software, running experiments to improve efficiency, and coordinating incident responses during launches.