Senior AI Evaluation Engineer

Distyl; New York (NY); Hybrid; USD 150,000 - 250,000; Full time

Benefits offered by this job

100% covered medical, dental, and vision

401(k) with additional perks

Access to modern AI tools

Job summary

Distyl in New York is seeking an AI Evaluation Engineer to design and implement frameworks for evaluating AI systems. The ideal candidate will have 2+ years of software engineering experience and strong Python skills.

This role offers a base salary range of $150K to $250K, comprehensive benefits, and equity opportunities, with a hybrid work model requiring in-office attendance 3+ days per week.

Qualifications

• 2+ years of software engineering experience.
• Write clean, maintainable Python and build evaluation pipelines.
• Understand evaluation and experimentation frameworks.

Responsibilities

• Design and implement evaluation frameworks for AI systems.
• Define system quality metrics reflecting user needs.
• Build and maintain golden test cases in Python.

Skills

Strong Python Engineering Skills

Experience with Evaluation-Driven Development

Systems-Oriented Mindset

Ability to Translate Human Judgment into Code

AI-Native Working Style

Job description

Distyl in New York is seeking an AI Evaluation Engineer to design and implement frameworks for evaluating AI systems. The ideal candidate will have 2+ years of software engineering experience and strong Python skills.

This role offers a base salary range of $150K to $250K, comprehensive benefits, and equity opportunities, with a hybrid work model requiring in-office attendance 3+ days per week.