Remote AI Evaluation Platform Lead

Jobgether SRL; Italia; Remote; EUR 90,000 - 120,000; Full time

Benefits offered by this job

Competitive compensation + equity

Comprehensive health coverage

Parental leave

Job summary

Jobgether SRL is seeking an Evaluations Team Lead in Italy to shape how AI models are evaluated and deployed in production. You will lead a team to build a shared evaluation platform, set rigorous benchmarking standards, and drive data-driven research priorities across research, engineering, and customer-facing teams.

You will manage a small team, stay hands-on with Python development, and influence product releases by evaluating predictive quality, latency, and cost efficiency.

Qualifications

• Proven experience owning evaluation of ML models in production or high‑impact decisions.
• Strong statistical expertise: metrics, validation, uncertainty estimation.
• Experience preventing data leakage across train/eval splits (preprocessing, joins).
• Proficient in Python and SQL; familiarity with ML tooling used by multiple teams.
• Experience designing fair, reproducible model comparisons with end‑to‑end latency focus.

Responsibilities

• Build and maintain a shared evaluation platform for research, engineering, and Applied AI teams.
• Develop benchmark suites, datasets, and model comparisons for predictive quality across use cases.
• Establish evaluation standards: metrics, validation, data splits, leakage prevention, calibration, uncertainty estimation.

Skills

Statistical analysis

Python

SQL

People management

Experiment design

Data quality

Tools

scikit-learn

Snowflake

Databricks

CI/CD pipelines

Job description

Jobgether SRL is seeking an Evaluations Team Lead in Italy to shape how AI models are evaluated and deployed in production. You will lead a team to build a shared evaluation platform, set rigorous benchmarking standards, and drive data-driven research priorities across research, engineering, and customer-facing teams.

You will manage a small team, stay hands-on with Python development, and influence product releases by evaluating predictive quality, latency, and cost efficiency.