Remote AI Research Engineer — Build & Evaluate Agents

Remote; San Francisco (CA); Remote; USD 140,000 - 200,000; Full time

Benefits offered by this job

Remote first

In-person retreats

Paid medical/dental/vision

WFH stipend

Home internet stipend

Unlimited vacation

Parental leave

Fertility benefits

Incentive compensation

Job summary

Gauntlet is hiring an AI Research Engineer to improve how we build, evaluate and use AI agents across the business. You’ll create evaluation frameworks, run experiments, and push improvements from research into production with Aria and other agents.

The role blends engineering with open-ended research and long-term system development. The position offers a remote-first setup for candidates in the United States, with strong emphasis on experiments, tooling, and cross-team collaboration to advance

Qualifications

• Hands-on experience building, evaluating or improving LLM models or agents.
• Strong Python skills and production-grade coding.
• Experience in applied research, ML or data science.
• Understanding of LLM agents: tool use, harnesses and prompting fail points.
• Ability to break open-ended problems into testable questions.

Responsibilities

• Build evaluation frameworks for AI agents with datasets, sandboxes and metrics.
• Design experiments comparing models, harnesses and environments; analyze trade-offs.
• Collaborate with Gauntlet teams to convert feedback into experiments and outcomes.
• Investigate agent failures and improve performance across components.
• Develop and improve agent harnesses, tools and integrations (Aria and others).
• Contribute to shared knowledge and reusable agent skill systems.
• Work with Infrastructure and Security to ensure safe production usage.

Skills

Python

LLM Agents

Applied ML

Problem Framing

Tooling Dev

Cross-functional

Education

Advanced degree in CS/ML

Tools

Hermes/OpenClaw

Job description

Gauntlet is hiring an AI Research Engineer to improve how we build, evaluate and use AI agents across the business. You’ll create evaluation frameworks, run experiments, and push improvements from research into production with Aria and other agents.

The role blends engineering with open-ended research and long-term system development. The position offers a remote-first setup for candidates in the United States, with strong emphasis on experiments, tooling, and cross-team collaboration to advance