AI Risk & Fraud Evaluation Engineer

Variance; San Francisco (CA); On-site; USD 170,000 - 230,000; Full time

Benefits offered by this job

Competitive salary

Platinum-level medical, dental, and vision insurance

Unlimited PTO

$100 reimbursement for wellness expenses

401(k) plan

Job summary

A technology firm in San Francisco is seeking a Research Engineer to enhance AI model quality. The ideal candidate will build benchmarks, datasets, and evaluation loops to ensure effective performance on critical tasks. This role requires strong programming skills and a passion for developing rigorous evaluations in high-stakes environments. A commitment to craftsmanship and deep understanding of ML systems are essential. The company offers competitive salary and robust benefits.

Qualifications

• Experience training, evaluating, or improving modern ML systems.
• Strong programming skills and comfort working in research-heavy codebases.
• Experience building benchmarks, datasets, evaluation pipelines, or quality systems.

Responsibilities

• Build proprietary benchmarks and datasets to evaluate models.
• Design and run evaluations that measure model performance.
• Define quality metrics for judgment systems.

Skills

Strong programming skills

Experience with ML systems

Engineering judgment

Interest in fraud and risk

Job description

A technology firm in San Francisco is seeking a Research Engineer to enhance AI model quality. The ideal candidate will build benchmarks, datasets, and evaluation loops to ensure effective performance on critical tasks. This role requires strong programming skills and a passion for developing rigorous evaluations in high-stakes environments. A commitment to craftsmanship and deep understanding of ML systems are essential. The company offers competitive salary and robust benefits.