Remote Research Engineer — AI Evaluation & RL Data Pipelines

Prime Intellect AI; San Francisco (CA); Hybrid; USD 150,000 - 350,000; Full time

Benefits offered by this job

Salary + equity

Remote or SF Office

Visa sponsorship

Team off-sites & learning

Mission-driven team

Job summary

Prime Intellect AI is building an open frontier AI stack that unifies training, evaluation, and deployment for long-horizon tasks. We seek engineers to design challenging evaluations and scalable RL data pipelines, working closely with researchers to push model capabilities and measure progress with robust scoring and verifiers.

The role emphasizes strong Python software engineering, data pipeline construction, and hands-on experience with language models or agents.

Qualifications

• Strong Python software engineering skills for data pipelines, evaluation systems, or ML workflows.
• Hands-on experience with language models or agents and understanding training data impact on behavior.
• Ability to inspect complex data and model outputs to identify failure modes and tests.
• Proven track record delivering technical projects from experiment to validation in collaboration with researchers.
• Experience building evaluations or agent environments with traceable debugging of harness and scoring code.

Responsibilities

• Design and build challenging evaluations for coding and long-horizon agents.
• Build executable environments in Verifiers with realistic task setup and clear success criteria.
• Develop and test scoring logic and verifiers, investigating reward hacks.
• Trace failures through model outputs to distinguish model limitations from environment issues.
• Build and scale RL data pipelines that generate coding and agentic tasks.
• Collaborate with RL researchers to test task/training data changes on held-out tasks.

Skills

Python

Data pipelines

Evaluation systems

RL research collaboration

Model/agent introspection

Tools

Verifiers

RL frameworks

Job description

Prime Intellect AI is building an open frontier AI stack that unifies training, evaluation, and deployment for long-horizon tasks. We seek engineers to design challenging evaluations and scalable RL data pipelines, working closely with researchers to push model capabilities and measure progress with robust scoring and verifiers.

The role emphasizes strong Python software engineering, data pipeline construction, and hands-on experience with language models or agents.