AI Evaluation Platform Lead

Jobgether; United States; Remote; USD 150.000 - 230.000; Vollzeit

Benefits dieser Stelle

Relocation assistance

Remote work across Europe and Israel

Equity and competitive compensation

Zusammenfassung

Jobgether is seeking an Evaluations Team Lead to shape how enterprise AI models are evaluated and improved. You will build a shared platform, define rigorous benchmarking standards, and drive data-driven decisions across research, engineering, and customer teams.

You will manage a small team, stay hands-on with Python and statistics, and collaborate across research, engineering, and customer data science functions to deliver reliable, scalable evaluation pipelines.

Qualifikationen

• Proven experience owning evaluation of tabular ML in production or high-impact customer decisions.
• Strong statistical expertise in metrics, validation schemes, and uncertainty estimation.
• Experience identifying and preventing data leakage across training and evaluation splits.
• Proficiency in Python and SQL; familiarity with scikit-learn and gradient-boosted trees.
• Previous people management experience including hiring, mentoring, and performance feedback.
• Excellent written and verbal communication; ability to collaborate with researchers, engineers, and data scientists.

Aufgaben

• Build and maintain a shared evaluation platform for research, engineering, and Applied AI teams.
• Develop and improve benchmark suites, datasets, and model comparisons across business use cases.
• Establish evaluation standards for metrics, validation, data splits, leakage prevention, calibration, and uncertainty.
• Design reproducible evaluation pipelines with version-controlled inputs and artifacts, with regression checks in release workflows.
• Benchmark models with fair protocols for hyperparameter tuning, resources, and end-to-end latency.
• Evaluate predictive performance, latency, and cost across configurations and dataset characteristics.
• Support customer proof-of-concept evaluations with tooling and data-driven analysis.
• Translate findings into priority research areas and evidence-based improvement plans.
• Recruit, lead, and develop a small team of evaluation specialists with coaching and feedback.
• Remain hands-on with Python development, experiments, statistics, and methodology.

Jobbeschreibung

Jobgether is seeking an Evaluations Team Lead to shape how enterprise AI models are evaluated and improved. You will build a shared platform, define rigorous benchmarking standards, and drive data-driven decisions across research, engineering, and customer teams.

You will manage a small team, stay hands-on with Python and statistics, and collaborate across research, engineering, and customer data science functions to deliver reliable, scalable evaluation pipelines.