Evaluation-Driven AI Engineer (Python, Production)

Distyl; San Francisco (CA); Hybrid; USD 150,000 - 250,000; Full time

Benefits offered by this job

Medical, dental, and vision coverage

401(k) with commuter benefits

Access to modern AI tools

Job summary

Distyl is seeking an AI Evaluation Engineer to design evaluation systems for AI-driven solutions. Located in San Francisco, this position requires strong Python skills and a commitment to utilizing AI tools to enhance evaluation processes.

The role offers a competitive salary range of $150K – $250K based on experience, with comprehensive benefits including medical coverage and 401(k) plans. You will be working in a hybrid model with 3+ days per week in the office.

Qualifications

• 2+ years of software engineering experience.
• Strong Python engineering skills, clean and maintainable code.
• Experience with Evaluation-Driven Development frameworks.

Responsibilities

• Design and implement evaluation frameworks for AI systems.
• Define metrics and build test cases in Python.
• Develop evaluation pipelines integrating directly into system iteration loops.

Skills

Python engineering skills

Understanding Evaluation-driven Development

Systems-oriented mindset

AI-native working style

Job description

Distyl is seeking an AI Evaluation Engineer to design evaluation systems for AI-driven solutions. Located in San Francisco, this position requires strong Python skills and a commitment to utilizing AI tools to enhance evaluation processes.

The role offers a competitive salary range of $150K – $250K based on experience, with comprehensive benefits including medical coverage and 401(k) plans. You will be working in a hybrid model with 3+ days per week in the office.