Research Engineer - Gauntlet

Gauntlet; United States; Remote; USD 90,000 - 130,000; Full time

Job summary

Gauntlet is seeking a Research Engineer to advance AI agent evaluation in finance. You will build frameworks and tools for evaluating AI systems, with a strong emphasis on Python and LLMs.

You will design experiments, create datasets and metrics to assess performance on real financial tasks, and collaborate with cross-functional teams to translate qualitative feedback into measurable research outcomes. This role also involves diagnosing failures and improving internal tooling.

Qualifications

• Experience with Python and large language models for AI agent evaluation.
• Ability to design experiments and evaluate AI systems in finance.

Responsibilities

• Develop evaluation frameworks for AI agents.
• Create datasets and metrics to assess performance on real financial tasks.
• Collaborate with cross-functional teams to translate qualitative feedback into measurable research outcomes.
• Investigate agent failures and improve internal tools.

Skills

Python

LLMs

Experiment design

Data analysis

Job description

Gauntlet is looking for a Research Engineer to enhance AI agent evaluation in finance. This role focuses on building frameworks and tools for AI systems, requiring strong Python skills and experience with LLMs. Ideal candidates are research-oriented engineers with a passion for AI agents.

In this role, you will develop evaluation frameworks for AI agents, which includes creating datasets and metrics to assess performance in real-world financial tasks. Your work will involve designing experiments to compare models and tools, identifying regressions, and understanding performance trade-offs.

Collaboration is key at Gauntlet, as you will work with various teams to translate qualitative feedback into measurable research outcomes. You will also investigate agent failures and enhance internal tools, contributing to the continuous improvement of AI agents used across the organization.