AI Quality & Evaluation Data Scientist (Product)

DutchTechX; Kraków; Hybrid; PLN 120,000 - 180,000; Full time

Job summary

Finom is expanding its AI team to own evaluation loops for all AI products, including an offline eval suite and online dashboards. You will work with AI engineers and product experts to ensure robust quality signals and actionable insights that drive product improvements across Europe.

The role emphasizes rigorous statistics, end-to-end data analysis with Python/SQL, and collaboration in a fast-paced startup environment.

Qualifications

• 3+ years in analyst / data scientist roles, at least one in a product context.
• Proficiency in Python and SQL for end-to-end analysis.
• Solid foundation in statistics: sampling, hypothesis testing, variance.
• Analytical mindset focusing on the business question, not just the tool.

Responsibilities

• Own offline eval suite across products: datasets, judges, metrics.
• Build and maintain online quality dashboards: resolution rate, CSAT, LLm signals, latency.
• Mine failure patterns from real traffic and translate into regression cases for product teams.
• Harden methodology: judge stability and non-determinism handling.
• Translate numbers into decisions with weekly syncs and trade-off clarity.

Skills

Python

SQL

Statistics

Analytical mindset

Product context

Tools

Claude Code

Job description

Finom is expanding its AI team to own evaluation loops for all AI products, including an offline eval suite and online dashboards. You will work with AI engineers and product experts to ensure robust quality signals and actionable insights that drive product improvements across Europe.

The role emphasizes rigorous statistics, end-to-end data analysis with Python/SQL, and collaboration in a fast-paced startup environment.