Data Infrastructure Engineer — Scalable ML Data Pipelines | Flexible Hours

JobCubby; San Francisco, Northern (CA, KY); Hybrid; USD 500,000 - 850,000; Full time

Benefits offered by this job

Visa sponsorship

Office in San Francisco

Flexible working hours

Job summary

Anthropic in San Francisco is seeking a Staff Engineer for the Pre-training team to advance the next generation of LLMs. You will design scalable data processing infrastructure, implement core primitives, and partner with researchers to ensure safety and efficiency in model training.

The role emphasizes distributed systems, high throughput, and collaboration across ML, data, and engineering disciplines. A strong background in CS and ML infra is required, with a passion for reliable AI.

Qualifications

• Bachelor’s degree in CS or related field.
• 5+ years of engineering experience outside internships.
• Experience in building distributed, high-throughput systems.
• Strong Python and Rust experience.
• Experience with language model training infrastructure.

Responsibilities

• Design data processing infrastructure for large language model training.
• Develop and maintain core processing primitives with a focus on scalability.
• Build data quality assurance and validation systems at scale.
• Collaborate with research teams to implement data processing architectures.
• Build end-to-end data pipelines turning raw corpora into training-ready datasets.

Skills

Distributed systems

High-throughput

Data infrastructure

MLOps

Collaboration

Education

Bachelor’s degree in Computer Science or related field

Advanced degree in CS or related field

Tools

Apache Spark

Python

Rust

Job description

Anthropic in San Francisco is seeking a Staff Engineer for the Pre-training team to advance the next generation of LLMs. You will design scalable data processing infrastructure, implement core primitives, and partner with researchers to ensure safety and efficiency in model training.

The role emphasizes distributed systems, high throughput, and collaboration across ML, data, and engineering disciplines. A strong background in CS and ML infra is required, with a passion for reliable AI.