AI Engineer, RL & Evals

raydar; California; On-site; USD 175,000 - 275,000; Full time

Benefits offered by this job

Competitive equity

Job summary

raydar seeks an AI Engineer focused on reinforcement learning and evaluations to build production systems that help models improve on subjective tasks. You will design RL environments, evaluation frameworks, and agent harnesses, collaborating with research teams to ship backend and data pipelines.

The role requires strong Python and PyTorch skills, RL/LLMs exposure, and a CS degree. On-site at our SF office, with generous equity and impact on cutting-edge AI evaluation infrastructure.

Qualifications

• Requires a CS degree and practical software development experience.
• Proficient in Python and PyTorch for ML workflows.
• Experience building RL environments and evaluation systems.
• Familiarity with distributed systems and backend services.
• English communication across engineering and research teams.

Responsibilities

• Build reinforcement learning environments and evaluation frameworks for creative tasks.
• Design tasks and rubrics to measure quality in subjective domains.
• Develop agent harnesses and context layers for API and product infra.
• Collaborate with research to create environments improving model output.
• Ship production software end-to-end, including data pipelines.

Skills

Python

PyTorch

Reinforcement learning

LLMs

Distributed systems

Backend engineering

Education

Bachelor's degree in Computer Science

Job description

About the company

Our client is a seed-stage AI company building the data and infrastructure layer to improve how AI models handle subjective work. Its initial focus is design, where it develops reinforcement learning environments and evaluation systems for creative output. The team works with frontier AI labs and application companies on training, evaluation, and product infrastructure.

The role / why it matters

As an AI Engineer focused on reinforcement learning and evaluations, you will build the systems that help models improve on tasks where there is no single objectively correct answer. This is a production engineering role at the intersection of backend development and applied machine learning, with ownership from environment design through shipped infrastructure.

What you'll do

• Build and scale reinforcement learning environments and evaluation frameworks for creative domains such as design.
• Design tasks and grading rubrics that measure quality in subjective domains.
• Develop agent harnesses and context layers for API and product infrastructure.
• Work with research teams to create environments that improve AI model output.
• Ship production software end to end, including backend systems and data pipelines.

What we're looking for

• At least two years of relevant engineering or applied machine learning experience.
• Experience building reinforcement learning environments, evaluation systems, or machine learning post-training systems.
• Strong backend or full-stack engineering experience shipping production products.Practical skills in Python and PyTorch, with working knowledge of reinforcement learning, LLMs, evaluation frameworks, and distributed systems.
• A computer science degree.
• Ability to collaborate across engineering and research and communicate fluently in English.

Bonus points

• Experience building evaluations for creative, generative, or otherwise non-verifiable domains.
• Experience developing platforms or environments for agentic systems.

Compensation and benefits

The base salary range is $175,000 to $275,000 USD, plus competitive equity.

Location / work model

This is a full-time, on-site role in San Francisco, five days per week.