Remote AI Benchmark & Datasets Engineer (Contract)

deepsense.ai; Poland; Remote; PLN 201,000 - 335,000; Full time

Job summary

deepsense.ai is seeking an AI Benchmark & Datasets Engineer to design and build benchmarks for evaluating frontier AI models. You will focus on Exploratory Data Analysis and Data Science reasoning, while exploring what should come next in model evaluation.

Datasets may be synthetic or based on real data. The role emphasizes designing problems that require genuine reasoning and reliably separate stronger models from weaker ones, with remote/work-from-home flexibility on a contract basis.

Qualifications

• Solid understanding of Data Science and Exploratory Data Analysis, with hands-on experience working with data.
• Working knowledge of Python and the Data Science ecosystem.
• Working knowledge of Git.
• Good written English, at least B2.

Responsibilities

• Design original benchmark problems (EDA, Data Science, and related analytical domains) and build the datasets behind them, synthetic or real.
• Define ground truth, expected outputs, and evaluation criteria; write clear, reproducible task specifications.
• Test problems against current models, analyze failure modes and shortcuts, and calibrate difficulty.
• Review and evaluate benchmark problems created by other contributors.
• Investigate new classes of problems and model capabilities worth testing — and document everything in English.

Skills

Data Science

Exploratory Data Analysis

Analytical thinking

Creative problem design

Tools

Python

Git

Job description

deepsense.ai is seeking an AI Benchmark & Datasets Engineer to design and build benchmarks for evaluating frontier AI models. You will focus on Exploratory Data Analysis and Data Science reasoning, while exploring what should come next in model evaluation.

Datasets may be synthetic or based on real data. The role emphasizes designing problems that require genuine reasoning and reliably separate stronger models from weaker ones, with remote/work-from-home flexibility on a contract basis.