AI Quality & Evaluation Data Scientist

Finom; Kraków; Hybrid; PLN 180,000 - 240,000; Full time

Job summary

Finom is expanding its AI team across Europe, seeking a data-savvy analyst or data scientist to own the eval loop for AI products. You will partner with AI Quality, engineers, and product teams to improve datasets, metrics, and decision quality.

The role emphasizes end-to-end analytics, dashboards, and actionable insights to drive business impact. We value strong statistics, Python/SQL skills, and a product-focused mindset.

Qualifications

• 3+ years in analyst / data scientist roles, at least one in a product context.
• Python and SQL — you can build an analysis end-to-end.
• Solid foundation in statistics — sampling, hypothesis testing, variance.
• Analytical mindset — you start from the business question, not from the tool.

Responsibilities

• Own and extend offline eval suite across products — datasets (capability + regression), judges, metrics.
• Build and maintain online quality dashboards: resolution rate, CSAT, thumbs up/down, LLM-as-judge signals, error rate, latency.
• Close production feedback loop: mine failure patterns from real traffic, turn into regression cases, propose fixes to Product and domain experts.
• Harden methodology: judge stability, non-determinism handling.
• Translate numbers into decisions – weekly syncs, clear trade-offs.

Job description

Finom is expanding its AI team across Europe, seeking a data-savvy analyst or data scientist to own the eval loop for AI products. You will partner with AI Quality, engineers, and product teams to improve datasets, metrics, and decision quality.

The role emphasizes end-to-end analytics, dashboards, and actionable insights to drive business impact. We value strong statistics, Python/SQL skills, and a product-focused mindset.