AI Research Engineer — Evaluation Frameworks (Remote)

Gauntlet; United States; Remote; USD 175,000 - 195,000; Full time

Benefits offered by this job

Remote work from anywhere in US

Company retreats

Medical, dental, vision insurance

WFH stipend

Internet/Home cell reimbursement

Unlimited vacation

Parental leave (12 weeks)

Equity opportunity

Job summary

Gauntlet is hiring an AI Research Engineer to enhance how we build, evaluate, and deploy AI agents across the business. You will design evaluation frameworks, run real-world experiments, and translate findings into production-ready improvements.

The role blends applied research with engineering and offers opportunities to influence Aria and other agents. We seek hands-on researchers who can work independently, design rigorous experiments, and collaborate across teams to scale robust agent

Qualifications

• Experience building, evaluating or improving LLM models or agents.
• Strong Python and software engineering fundamentals.
• Ability to write production-quality code and test software.
• Applied ML or data science experience with experiments and datasets.
• Understanding of LLM agents, tool use, context management and failure modes.

Responsibilities

• Build evaluation frameworks for AI agents with datasets, sandbox environments and metrics.
• Design experiments to compare models, agents and tooling; identify regressions and trade-offs.
• Translate qualitative feedback into research questions and measurable outcomes.
• Investigate agent failures and improve performance across models, harnesses and prompting.
• Contribute to Aria and other internal agents; ship integrations and tooling.
• Develop shared knowledge and reusable agent skills for the team.
• Collaborate with Infrastructure and Security to ensure safe production operation.

Skills

Python

Experiment design

Research

Agent evaluation

Independent work

Collaboration

AI safety

Education

Advanced degree in CS/Math/ML

Tools

Hermes

OpenClaw

Job description

Gauntlet is hiring an AI Research Engineer to enhance how we build, evaluate, and deploy AI agents across the business. You will design evaluation frameworks, run real-world experiments, and translate findings into production-ready improvements.

The role blends applied research with engineering and offers opportunities to influence Aria and other agents. We seek hands-on researchers who can work independently, design rigorous experiments, and collaborate across teams to scale robust agent