AI Product Evaluation Lead – Banking Operations

LE001 Berkeley Research Group, LLC; United States; Remote; USD 180,000 - 250,000; Full time

Job summary

BRG is hiring an AI Product Engineer focused on evaluation to verify AI products for banking operations. You will build automated tests, investigate failures, and provide evidence to support release decisions while collaborating with banking experts and engineers on product improvements.

You will define correct results, measure performance, and guide testing from early development through live use, ensuring responsible and accurate behavior in complex financial workflows.

Qualifications

• 5+ years of experience in AI evaluation and product testing.
• Ownership of AI evaluations including test design, scoring, and failure analysis.
• Strong Python skills and automated test software development experience.
• Ability to interpret results, manage risk, and communicate findings clearly.

Responsibilities

• Define evaluation approach for AI products and build reusable tests that measure accuracy, reliability, and safety.
• Reproduce failures with engineers, verify fixes, and contribute to product improvements.
• Measure and report results with sample sizes, uncertainty, and comparison across versions.

Skills

Python

AI evaluation

Test automation

Release decisions

Data validation

Job description

BRG is hiring an AI Product Engineer focused on evaluation to verify AI products for banking operations. You will build automated tests, investigate failures, and provide evidence to support release decisions while collaborating with banking experts and engineers on product improvements.

You will define correct results, measure performance, and guide testing from early development through live use, ensuring responsible and accurate behavior in complex financial workflows.