RL Evaluation Engineer for Frontier AI & Data

Wwshemi; San Francisco (CA); Remote; USD 150,000 - 350,000; Full time

Benefits offered by this job

Remote work

SF office access

Visa sponsorship

Relocation assistance

Hackathons & off-sites

Learning opportunities

Job summary

Prime Intellect is building the open frontier AI platform that unifies data, environments, evaluations and deployments for frontier-scale research. Join the role focused on designing challenging evaluations and scalable RL tasks for coding and long-horizon agents, collaborating directly with researchers to push model capabilities and reliability.

We offer a cash compensation range of $150-350k with equity, plus flexible work arrangements including remote or SF office, visa sponsorship and

Qualifications

• Strong software engineering skills in Python and data pipelines.
• Hands-on experience with language models or agents.
• Ability to inspect complex data and model outputs, identify failure modes, and turn an ambiguous quality question into a testable measurement.
• A record of owning technical projects through experimentation, implementation, and validation, while collaborating closely with researchers and engineers.
• Hands-on experience building technical evaluations or agent environments, with the engineering depth to inspect model traces and debug harness and scoring code.

Responsibilities

• Design and build challenging evaluations for coding and long-horizon agents.
• Build executable environments in Verifiers, with realistic task setup and clear success criteria.
• Develop and test scoring logic and verifiers, investigating reward hacks and misleading measures of progress.
• Trace failures through model outputs and harness code to distinguish model limitations from problems in the environment or scoring.
• Build and scale RL data pipelines that generate and curate challenging coding and agentic tasks.
• Work with RL researchers to test whether changes to tasks and training data improve performance on held-out tasks.

Skills

Python

ML workflows

Data pipelines

Evaluation systems

Research collaboration

Language models

Tools

Verifiers

Job description

Prime Intellect is building the open frontier AI platform that unifies data, environments, evaluations and deployments for frontier-scale research. Join the role focused on designing challenging evaluations and scalable RL tasks for coding and long-horizon agents, collaborating directly with researchers to push model capabilities and reliability.

We offer a cash compensation range of $150-350k with equity, plus flexible work arrangements including remote or SF office, visa sponsorship and