RL Engineer for Subjective AI: Evals & Environments

Tastelabs; San Francisco, Northern (CA, KY); Hybrid; USD 120,000 - 190,000; Full time

Benefits offered by this job

Open source contributions

Job summary

Taste Labs in San Francisco is building the data and infrastructure layer for taste, advancing evaluation methods for subjective design tasks and enabling frontier models to perform with context and verifiability.

You will design tasks that capture elements of taste, build RL infrastructure, and work with research teams to craft environments that improve model behavior. A startup mindset and backend skills are essential.

Qualifications

• Experience designing evals and grading schemes for non-verifiable outputs.
• Experience building RL environments and backend systems for data pipelines.
• Strong backend development skills and ability to ship in a startup setting.

Responsibilities

• Research grading methods and rubrics for subjective domains.
• Design tasks capturing elements of “taste” and design capabilities.
• Build agent harnesses and context layers for evaluation pipelines.
• Work on scalable RL infrastructure and data pipelines.
• Collaborate with internal research teams on training pipelines and evaluation methods.

Skills

Eval design

RL environments

Backend development

Startup experience

Tools

Python

Job description

Taste Labs in San Francisco is building the data and infrastructure layer for taste, advancing evaluation methods for subjective design tasks and enabling frontier models to perform with context and verifiability.

You will design tasks that capture elements of taste, build RL infrastructure, and work with research teams to craft environments that improve model behavior. A startup mindset and backend skills are essential.