Data Scientist

Slashhash; Amsterdam; Hybrid; EUR 60,000 - 90,000; Full time

Job summary

Kaizo is seeking a Data Scientist to own the evaluation and quality loop of their AutoQA product, which uses LLMs to review customer support conversations. You will translate customer rubrics into testable instructions, run experiments, build datasets and make results actionable.

The ideal candidate has strong Python skills, applied statistics knowledge and 0-2 years of industry experience, with a focus on experimental design and classifier evaluation.

Qualifications

• Strong Python skills and applied statistics knowledge.
• 0-2 years of industry experience.
• Ability to translate customer rubrics into testable instructions and run experiments.

Responsibilities

• Translate customer rubrics into AutoQA instructions.
• Run experiments to improve accuracy using LangSmith and BigQuery.
• Build golden datasets and generate synthetic data.
• Build LLM-as-a-judge pipelines.
• Drive decisions on prompts, tools and capabilities to make results actionable.
• Evaluate retrieval, tool calling, and speech pipelines.
• Join customer calls to understand quality definitions.

Skills

Python

Pandas

NumPy

Jupyter

Applied Statistics

Experiment Design

Classifier Evaluation

Precision/Recall Trade-offs

Imbalanced Data

LLMs

Prompt Engineering

LangSmith

BigQuery

SQL

Tools

Docker

Kubernetes

Google Cloud Platform

Job description

Kaizo is seeking a Data Scientist to own the evaluation and quality loop of their AutoQA product, which uses LLMs to review customer support conversations. The role involves translating customer rubrics into testable instructions, running experiments, building datasets, and making results actionable. Candidates should have strong Python skills, applied statistics knowledge, and 0-2 years of industry experience.

Technical (Must-have)

• Python
• Pandas
• NumPy
• Jupyter
• Applied Statistics
• Experiment Design
• Classifier Evaluation
• Precision/Recall Trade-offs
• Imbalanced Data
• LLMs
• Prompt Engineering
• LangSmith
• BigQuery
• SQL

Soft Skills

• Analytical Mindset
• Product Sense
• Empathy
• Communication
• Team Player
• Resourcefulness

Technical (Nice-to-have)

• LangSmith
• Google Cloud Platform
• BigQuery
• Docker
• Kubernetes
• Speech Processing
• ASR Evaluation
• Fine-tuning LLMs
• Synthetic Datasets

Key Responsibilities

• Translate customer rubrics into AutoQA instructions
• Run experiments to improve accuracy using LangSmith and BigQueryli>
• Build golden datasets and generate synthetic data
• Build LLM-as-a-judge pipelines
• Make results actionable by driving decisions on prompts, tools, and capabilities
• Evaluate retrieval, tool calling, and speech pipelines
• Join customer calls to understand quality definitions

Python, Pandas, NumPy, Jupyter, Applied Statistics, Experiment Design, Classifier Evaluation, Precision/Recall Trade-offs, Imbalanced Data, LLMs, Prompt Engineering, LangSmith, BigQuery, SQL, 2 years of relevant experience