Research Engineer

Remote

San Francisco (CA)

Remote

USD 140,000 - 200,000

Full time

2 days ago

Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Remote first

In-person retreats

Paid medical/dental/vision

WFH stipend

Home internet stipend

Unlimited vacation

Parental leave

Fertility benefits

Incentive compensation

Job summary

Gauntlet is hiring an AI Research Engineer to improve how we build, evaluate and use AI agents across the business. You’ll create evaluation frameworks, run experiments, and push improvements from research into production with Aria and other agents.

The role blends engineering with open-ended research and long-term system development. The position offers a remote-first setup for candidates in the United States, with strong emphasis on experiments, tooling, and cross-team collaboration to advance

Qualifications

Hands-on experience building, evaluating or improving LLM models or agents.
Strong Python skills and production-grade coding.
Experience in applied research, ML or data science.
Understanding of LLM agents: tool use, harnesses and prompting fail points.
Ability to break open-ended problems into testable questions.

Responsibilities

Build evaluation frameworks for AI agents with datasets, sandboxes and metrics.
Design experiments comparing models, harnesses and environments; analyze trade-offs.
Collaborate with Gauntlet teams to convert feedback into experiments and outcomes.
Investigate agent failures and improve performance across components.
Develop and improve agent harnesses, tools and integrations (Aria and others).
Contribute to shared knowledge and reusable agent skill systems.
Work with Infrastructure and Security to ensure safe production usage.

Skills

Python

LLM Agents

Applied ML

Problem Framing

Tooling Dev

Cross-functional

Education

Advanced degree in CS/ML

Tools

Hermes/OpenClaw

Job description

The role

Gauntlet is looking for an AI Research Engineer to help improve how we build, evaluate and use AI agents across the business.

We already have agents running in production, including Aria, our in-house generalist agent. They're used across research, engineering and business workflows, and we've learned a lot from seeing how they perform in practice. Some tasks work extremely well; others produce inconsistent results, and changes to models or tooling can introduce unexpected problems. Today, much of that feedback is still subjective. We want to build a more systematic way of understanding what works, what doesn't and why.

A major part of this role will be building the evaluation frameworks that allow us to measure agent performance on real tasks. You'll work with our engineers, quantitative researchers and other teams to understand where agents struggle, turn those observations into measurable experiments, and use the results to improve the systems we're building.

You'll also contribute directly to our existing agents and internal tooling. Initially, the work will be more engineering-focused as you get familiar with our systems, shipping improvements and integrations that teams can use. Over time, you'll take on more open-ended research, helping us develop better approaches to evaluation, agent behaviour and infrastructure.

We're also thinking beyond individual agents. One of our longer-term goals is to build a shared system of knowledge and reusable skills that our agents can draw from and improve over time. You'll play an important role in developing that infrastructure and how we evaluate its effectiveness.

This is an applied research role with a meaningful engineering component. We're looking for someone who enjoys investigating difficult problems, can design good experiments and is comfortable building and shipping the solutions that come out of them.

About Gauntlet

Gauntlet builds the financial systems of the future. While much of onchain finance is focused on point solutions, we operate across the entire stack to offer best-in-class vault products. Today we serve over $1.7B in client TVL across some of the largest fintechs/neobanks, protocols, exchanges, and capital allocators in crypto - and, increasingly, traditional asset management. Our team brings together traditional finance and crypto-native expertise to deliver durable, sophisticated products for institutional clients moving onchain.

Why this role is different

This is an opportunity to work at the intersection of AI, quantitative research and DeFi, tackling problems where advances in AI can have a direct impact on financial markets and onchain strategies.

You'll go beyond building individual agents, helping define how they're evaluated, how they learn from real-world tasks and how their capabilities scale across the business. With few established benchmarks in this space, you'll have the freedom to explore new approaches and take your ideas from research through to production, working directly with the teams putting them to use.

What you'll do;

Build evaluation frameworks for AI agents, including datasets, sandbox environments and metrics that measure performance on real-world tasks.
Design experiments to compare models, agent harnesses, tools and execution environments, identifying regressions and understanding the trade-offs between performance, reliability and cost.
Work with teams across Gauntlet to turn qualitative feedback about agent behaviour into concrete research questions, experiments and measurable outcomes.
Investigate agent failures, understand whether problems come from the model, harness, tooling or prompting, and develop ways to improve performance.
Build and improve agent harnesses, internal tools and integrations, including contributing to Aria and other agents used across the business.
Help develop shared knowledge and reusable skills across agents, creating systems that can learn from previous work and improve over time.
Work alongside Infrastructure and Security to ensure agents can operate reliably and safely within our production environment.

What we're looking for;

Hands‑on experience building, evaluating or improving LLM models or agents, whether through professional work, applied research, open‑source contributions or substantial personal projects.
Strong Python skills and good software engineering fundamentals. You're comfortable writing production-quality code and understand the importance of testing, maintainability and reliability.
Experience with applied research, machine learning or data science, including designing experiments, working with datasets and drawing conclusions from results.
A good understanding of how LLM agents work beyond prompting, including tool use, harnesses, context management and common failure modes.
The ability to take an open-ended problem, break it down into testable questions and work independently towards a solution.
Experience building tools or systems that other people use, with an understanding of how to evaluate their effectiveness beyond technical benchmarks.
Comfort working across research and engineering, collaborating with different teams and adapting as priorities change.
Genuine interest in AI agents and how they can be made more capable, reliable and useful.

Nice to have;

Experience building LLM evaluation pipelines, benchmarks, reinforcement learning environments or model testing infrastructure.
Contributions to open-source agent frameworks, harnesses, evaluation datasets or benchmarks.
Experience with agent frameworks such as Hermes, OpenClaw or similar tools.
Experience in applied ML research, model training, fine‑tuning or reinforcement learning.
An advanced degree or postgraduate research experience in computer science, mathematics, statistics, machine learning or a related field.
Experience in financial markets, quantitative research, DeFi or risk modelling.
Familiarity with AI security considerations, including access controls, prompt injection and data protection.

You don't need prior experience in crypto or DeFi. Strong research and engineering fundamentals matter more, and you'll work closely with people across Gauntlet who understand the financial systems and strategies our agents support.

We expect strong candidates to have experience working in a professional engineering or research environment, but we're more interested in what you've built and how you approach problems than a specific number of years on your CV.

Benefits & Perks;