Staff AI Research Engineer - Post-Training Agents

Goaly; Menlo Park, Northern (CA, KY); Hybrid; USD 150,000 - 230,000; Full time

Benefits offered by this job

Meals and office benefits

Visa sponsorship

Job summary

Goaly is seeking a research-engineering role owner for the experimental loop that turns a capable base model into a useful agent. You will design tasks and environments, prepare training and evaluation data, run reinforcement-learning experiments, diagnose model behavior, and convert results into better recipes and production models.

This is a research-engineering role; you will work with RL systems, training, inference, product, and domain experts, improving infrastructure and pipelines as

Qualifications

• Strong Python and software-engineering skills, ability to turn ambiguous ideas into reliable experimental systems.
• Hands-on experience training, fine-tuning, or evaluating modern language models, or a closely related ML research area.
• Solid understanding of deep learning and optimization, plus reinforcement-learning intuition.
• Excellent experimental judgment: controls, data inspection, metrics, reproducibility.
• Ability to debug across model behavior, data, code, and distributed infrastructure.

Responsibilities

• Design and run post-training experiments for agentic capabilities, including tool use, coding, reasoning, planning, long-horizon task completion, and recovery from failure.
• Prepare high-quality training and evaluation data: define task distributions, curate and filter examples, control contamination, balance difficulty, and build reproducible data-generation pipelines.
• Build realistic RL environments and task harnesses with clear interfaces, reliable resets, isolated execution, useful telemetry, and reward signals that are hard to game.
• Develop evaluations that measure both capability and reliability. Create regression suites, behavioral slices, error taxonomies, and dashboards that connect metrics to failures.
• Iterate on training recipes, including supervised warm starts, sampling strategies, reward design, verifiers, curricula, optimization choices, and reinforcement fine-tuning methods.
• Analyze trajectories and model behavior to find reward hacking, shortcut learning, and other failure modes; turn findings into targeted experiments.
• Improve the research workflow through better experiment configuration, rollout inspection, reproducibility, checkpoint evaluation, and automated comparison of runs.
• Partner with systems engineers to debug cross-layer problems in rollout inference, environment execution, distributed training.
• Translate successful research ideas into stable pipelines and help set the team's longer-term post-training roadmap.

Skills

Python

Software engineering

RL basics

Tools

PyTorch

JAX

DeepSpeed

Ray

Job description

Goaly is seeking a research-engineering role owner for the experimental loop that turns a capable base model into a useful agent. You will design tasks and environments, prepare training and evaluation data, run reinforcement-learning experiments, diagnose model behavior, and convert results into better recipes and production models.

This is a research-engineering role; you will work with RL systems, training, inference, product, and domain experts, improving infrastructure and pipelines as