Research Engineer: AI Agent Evaluation for Finance

Gauntlet; United States; Remote; USD 90,000 - 130,000; Full time

Job summary

Gauntlet is seeking a Research Engineer to advance AI agent evaluation in finance. You will build frameworks and tools for evaluating AI systems, with a strong emphasis on Python and LLMs.

You will design experiments, create datasets and metrics to assess performance on real financial tasks, and collaborate with cross-functional teams to translate qualitative feedback into measurable research outcomes. This role also involves diagnosing failures and improving internal tooling.

Qualifications

• Experience with Python and large language models for AI agent evaluation.
• Ability to design experiments and evaluate AI systems in finance.

Responsibilities

• Develop evaluation frameworks for AI agents.
• Create datasets and metrics to assess performance on real financial tasks.
• Collaborate with cross-functional teams to translate qualitative feedback into measurable research outcomes.
• Investigate agent failures and improve internal tools.

Skills

Python

LLMs

Experiment design

Data analysis

Job description

Gauntlet is seeking a Research Engineer to advance AI agent evaluation in finance. You will build frameworks and tools for evaluating AI systems, with a strong emphasis on Python and LLMs.

You will design experiments, create datasets and metrics to assess performance on real financial tasks, and collaborate with cross-functional teams to translate qualitative feedback into measurable research outcomes. This role also involves diagnosing failures and improving internal tooling.