Research Engineer, Post-Training

present of AI; San Francisco (CA); On-site; USD 180,000 - 280,000; Full time

Job summary

Cognition in San Francisco is seeking a leader for the post-training loop of coding agents, responsible for building training recipes, designing evals, and diagnosing why results look strange.

The role requires depth in preference learning and reward methods such as RLHF, plus solid statistics to tell true effects from noise. The ideal candidate has trained large models across many GPUs and has published or open-source ML work. Priorities may shift quickly, demanding clear judgment.

Qualifications

• Experience with RLHF or preference modeling projects
• Statistical rigor on experiments
• Published or open-source ML work

Responsibilities

• The role covers the whole post-training loop for coding agents: building training recipes, designing evals, and working out why results look strange.
• This job is the AI work itself, since the person shapes how an autonomous software agent learns, behaves and gets measured.
• Someone in this role would work on training and aligning an agent that plans, writes, debugs and deploys code, using very large compute.

Job description

The role covers the whole post-training loop for coding agents: building training recipes, designing evals, and working out why results look strange. It asks for real depth in preference learning and reward methods such as RLHF, plus solid statistics to tell true effects from noise or bugs. A strong candidate has trained large models across many GPUs, has original work to point to, and is comfortable when priorities change fast.

How it fits the AI era

This job is the AI work itself, since the person shapes how an autonomous software agent learns, behaves and gets measured. Tooling can speed up experiments and data generation, but deciding what a good eval looks like and whether a gain is real still takes human judgment. Understanding why a training run behaved oddly and carrying that lesson forward also stays a human skill.

How it builds AI experience

Someone in this role would work on training and aligning an agent that plans, writes, debugs and deploys code, using very large compute. To build toward it, run small post-training experiments on an open model for coding tasks, such as preference tuning or reward modeling. Build your own coding evals and write up what you learned, including where the numbers misled you.

Worth showing

• RLHF or preference modeling projects
• Statistical rigor on experiments
• Published or open-source ML work

This is our own read, written from the employer's posting and the company's Spotlight. The employer's posting is the source of truth for the role.

About Cognition

Applied AI lab building Devin, an autonomous software engineering agent that plans, writes, debugs, and deploys code