Post-Training ML Engineer: Alignment & Evaluation

Doist; San Francisco (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

Doist is seeking an accomplished ML engineer to join Devin's post-training and alignment efforts. This role blends research and hands-on engineering to shape how ambitious AI agents learn, evaluate, and interact with humans in long-horizon tasks.

You will develop post-training recipes, design meaningful evaluations, and advance techniques like RLHF, RLAIF, and constitutional approaches. The team values rigorous experimentation, systems thinking, and rapid iteration in a world-class AI lab

Qualifications

• Post-training experience in ML systems and alignment
• Ability to design and run end-to-end evaluation pipelines
• Experience with large-scale distributed training is a plus
• Strong fundamentals in probability, statistics, and ML theory
• Ability to interpret experimental data and distinguish signals from noise

Responsibilities

• Develop post-training recipes across datasets, training stages, and hyperparameters.
• Design evaluations that capture meaningful signals and progress.
• Investigate results and diagnose why training behaves unexpectedly.
• Apply RLHF, RLAIF, and constitutional methods to agent behavior.
• Scale experiments with data and compute, exploring new methodologies.

Skills

RLHF

RLAIF

Reward learning

Probability

Statistics

ML theory

Job description

Doist is seeking an accomplished ML engineer to join Devin's post-training and alignment efforts. This role blends research and hands-on engineering to shape how ambitious AI agents learn, evaluate, and interact with humans in long-horizon tasks.

You will develop post-training recipes, design meaningful evaluations, and advance techniques like RLHF, RLAIF, and constitutional approaches. The team values rigorous experimentation, systems thinking, and rapid iteration in a world-class AI lab