Senior AI Evaluation Engineer — Metrics & Data Pipelines

Sentry; San Francisco (CA); Hybrid; USD 240,000 - 280,000; Full time

Benefits offered by this job

Equity grants

Paid time off

Group health insurance coverage

Job summary

A leading software monitoring company is seeking a Senior Software Engineer on its AI/ML team to build evaluation infrastructure for measuring the performance of AI systems. This role involves designing datasets, creating benchmarks, and ensuring AI features behave reliably. Candidates should have 5+ years of experience, proficiency in Python and TypeScript, and a background in AI/ML. The position offers a hybrid work model and a salary range of $240,000 to $280,000.

Qualifications

• 5+ years of professional experience in software engineering.
• Experience with AI/ML systems and evaluation techniques preferred.
• Strong coding skills in Python and TypeScript.

Responsibilities

• Design and build robust evaluation frameworks for AI systems.
• Create and curate high-quality datasets and benchmarks.
• Build automated test harnesses and metrics pipelines.

Skills

Python

TypeScript

AI/ML testing and evaluation

Building testing infrastructure

Data quality pipelines

Education

Bachelor's degree in Computer Science or related field

Job description

A leading software monitoring company is seeking a Senior Software Engineer on its AI/ML team to build evaluation infrastructure for measuring the performance of AI systems. This role involves designing datasets, creating benchmarks, and ensuring AI features behave reliably. Candidates should have 5+ years of experience, proficiency in Python and TypeScript, and a background in AI/ML. The position offers a hybrid work model and a salary range of $240,000 to $280,000.