Staff Data Infra Engineer for LLM Training Pipelines

Anthropic; San Francisco (CA); On-site; USD 500,000 - 850,000; Full time

Benefits offered by this job

Office in San Francisco

Equity donation matching (optional)

Flexible working hours

Generous vacation and parental leave

Job summary

Anthropic is seeking a Staff level Engineer to join the Pre-training team in San Francisco. You will design and build data processing infrastructures for large language model training, integrating research ideas with scalable engineering practice.

You will collaborate with researchers to deploy high-throughput, fault-tolerant systems, implement data quality pipelines, and help shape safe, steerable AI through robust infrastructure and tooling.

Qualifications

• 5+ years of experience outside internships in distributed systems.
• Strong software engineering skills for high-throughput fault-tolerant systems.
• Experience with distributed computing frameworks and Spark.

Responsibilities

• Design and implement data processing infrastructure for large language model training.
• Develop and maintain core processing primitives with a focus on scalability.
• Build robust data quality assurance and validation pipelines at scale.
• Collaborate with research teams to implement novel data processing architectures.
• Build and operate end-to-end data pipelines turning raw web-scale corpora into training-ready datasets.

Skills

Distributed systems

Python

Rust

Spark

Communication

Problem-solving

Education

Bachelor's degree in Computer Science or related field

Advanced degree in Computer Science

Tools

Apache Spark

Job description

Anthropic is seeking a Staff level Engineer to join the Pre-training team in San Francisco. You will design and build data processing infrastructures for large language model training, integrating research ideas with scalable engineering practice.

You will collaborate with researchers to deploy high-throughput, fault-tolerant systems, implement data quality pipelines, and help shape safe, steerable AI through robust infrastructure and tooling.