Applied AI Research Engineer - RL Environments & Evaluation

HeyMilo AI; San Francisco (CA); On-site; USD 150,000 - 210,000; Full time

Job summary

HeyMilo AI is hiring a Research Engineer to join our Applied AI team in San Francisco. You’ll design and build reinforcement learning environments with verifiable rewards for real-world use cases, creating simulators, reward functions, and evaluation harnesses to measure model performance.

The role blends ML research with systems engineering, collaborating directly with founders and advisors to take a use case from problem definition to reproducible environments for training and evaluation.

Qualifications

• Master's degree or PhD in AI, ML, CS or related field.
• Solid grounding in reinforcement learning and LLM post-training (reward design, policy optimization, evaluation).
• Strong software engineering skills in Python and reproducible code.

Responsibilities

• Design and build RL environments for real-world use cases, including simulators and reward functions.
• Create verifiable evaluation harnesses with clean scoring and cost tracking.
• Run post-training experiments to validate learnable signals from environments.
• Package environments and results for reproducibility and publish evaluations.
• Contribute to research write-ups and determine next-use cases.

Skills

Reinforcement learning

LLM evaluation

Python

Experiment design

Research collaboration

Education

Master's degree or PhD in AI/ML/CS

Tools

Docker

Linux

Cloud environments

Job description

HeyMilo AI is hiring a Research Engineer to join our Applied AI team in San Francisco. You’ll design and build reinforcement learning environments with verifiable rewards for real-world use cases, creating simulators, reward functions, and evaluation harnesses to measure model performance.

The role blends ML research with systems engineering, collaborating directly with founders and advisors to take a use case from problem definition to reproducible environments for training and evaluation.