AI Research Engineer, LLMs and Large Scale Training

Harnham; United States; Hybrid; USD 180,000 - 240,000; Full time

Job summary

Harnham is partnering with an early-stage AI lab to recruit a senior ML infrastructure engineer. You will scale workloads across large GPU clusters and work on reducing training time and costs while building robust tooling used by researchers.

You will join a small, senior team with strong ownership from first build to production, and enjoy hybrid options in San Francisco or New York, with remote consideration for the right candidate.

Qualifications

• 5+ years building ML systems in production.
• Distributed training or large-scale ML infrastructure experience.
• Proven track record shipping infrastructure used by other teams.
• Python, plus PyTorch or JAX.

Responsibilities

• Scale workloads across large GPU clusters.
• Cut the time and cost of training and running models.
• Monitor and debug for large training runs.
• Evaluation tooling for large language models.

Skills

Distributed ML systems

Python

PyTorch or JAX

Production deployment

Education

Master’s or PhD in ML/CS

Job description

Every major AI lab is chasing the same milestone: AI that can do real research. This team is building it now, at startup speed.

Our client is an early stage AI lab backed by a top tier VC and founded by people from leading AI labs and universities. They’re developing AI systems capable of doing research work. Behind every result is infrastructure, and that’s where you come in.

The team is small and senior. Engineers and researchers sit on the same team, and what you build gets used straight away.

Location: Hybrid in San Francisco or New York preferred, with fully remote considered for the right person.

Where you’d focus

• Scaling workloads across large GPU clusters
• Cutting the time and cost of training and running models
• Monitoring and debugging for large training runs
• Evaluation tooling for large language models

Requirements

• 5+ years building ML systems used in production
• Distributed training or large scale ML infrastructure experience
• A record of shipping infrastructure other teams depend on
• Python, plus PyTorch or JAX

Preferred

• GPU performance, inference or CUDA work
• A Master’s or PhD in machine learning, computer science or a related field
• Startup or early stage company experience
• Real world experience beyond university or large company settings

Why it’s worth a conversation

• An early seat on one of the most ambitious goals in AI
• Your work shows up in every result the team ships
• Compute to match the ambition, plus meaningful equity
• Work next to top researchers

Best suited to people who

• Want more scope than a large company can offer
• Have shipped in a fast moving, small team environment
• Like owning a system from first build to production

About this search: We’re running this search exclusively and on a retained basis, working as an embedded part of the client’s hiring team. All candidates for these roles come through us, and the client’s name is shared after an initial conversation.