Artificial Intelligence Researcher - Post Training

Velocity Tech; San Francisco (CA); On-site; USD 120,000 - 180,000; Full time

Job summary

Velocity Tech in San Francisco develops evaluations and training environments for AI labs to test and improve models. The Research Scientist will optimize how much models learn from tasks, maximizing training signal per hour of data produced.

You will combine hands-on RL production work with input from frontier researchers to assess data quality, build new datasets, and shape task distributions. This role directly impacts product impact and revenue growth.

Qualifications

• Hands-on experience running RL in production at a frontier lab.
• Experience taking an LLM through post-training from start to finish.
• Ability to set research direction and build it themselves in a fast-paced environment.
• Genuine fascination with how models learn from data, and how to make every task teach them as much as possible.

Responsibilities

• Partner with researchers at AI labs to develop post-training approaches and better ways of measuring data quality.
• Build and run the internal post-training pipeline to quantify how much the company’s tasks improve models.
• Turn available datasets and domain experts into new data products.
• Finding ways to build compounding feedback loops into how the company works.
• Building agents that can analyse thousands of environments and millions of trajectories.
• Shaping and defining the task distributions for new datasets.
• Working with the procurement team to check that external data is worth refining.
• Designing scalable systems to ingest and assess datasets before purchase.
• Developing new methods for extracting signal from raw data.

Skills

RL in production

LLM post-training

data quality assessment

research direction

Job description

The company is a research-led startup building the evaluations and training environments that leading AI labs use to test and improve their models.

Since raising its seed round from a top-tier investor, the company has spent the past year creating coding evals used on some of the most widely used coding models. It has since expanded into several other specialist domains, and demand from labs now exceeds what it’s able to deliver, so the team is growing quickly.

Its data generation is designed to scale through technology rather than headcount, and every product it ships runs on a single platform that improves the more it’s used. The team is small and close-knit, with proven product-market fit, and this role will have a direct impact on growing revenue many times over.

The role

The Research Scientist will be responsible for understanding and improving how much models actually learn from the company’s tasks. The core goal is to get the most training signal out of every hour spent producing data. The successful candidate will combine their own hands-on experience with input from frontier researchers to assess data quality, identify where it falls short, and build new datasets. The role requires a background doing this kind of work in production at a frontier lab.

Responsibilities may include

• Partnering with customers, who are researchers at leading AI labs, to develop new post-training approaches and better ways of measuring data quality
• Building and running the internal post-training pipeline to quantify how much the company’s tasks improve models
• Turning available datasets and domain experts into new data products
• Finding ways to build compounding feedback loops into how the company works
• Building agents that can analyse thousands of environments and millions of trajectories
• Shaping and defining the task distributions for new datasets
• Working with the procurement team to check that external data is worth refining
• Designing scalable systems to ingest and assess datasets before purchase
• Developing new methods for extracting signal from raw data

The ideal candidate will have

• At least a year of hands-on experience running RL in production at a frontier lab
• Experience taking an LLM through post-training from start to finish
• The drive to set research direction and build it themselves in a fast-paced environment
• A genuine fascination with how models learn from data, and how to make every task teach them as much as possible