Senior AI Evaluation Data Scientist — Remote

Syllo; United States; Remote; USD 160,000 - 200,000; Full time

Benefits offered by this job

Health insurance

Equity

Job summary

Syllo in the United States is seeking a Senior Data Scientist to evaluate and improve AI system quality and performance. You'll design experiments, analyze production data, and partner with Engineering, Product, and Analytics to drive measurable improvements.

This hands-on role sits at the intersection of data science and applied AI, with emphasis on instrumentation, dashboards, and clear communication of findings to technical and non-technical stakeholders.

Qualifications

• Master's degree in computer science, data science, machine learning, or a related field, or 2–4 years of relevant professional experience.
• Strong Python and SQL skills and experience working directly with production data.
• Comfortable navigating production code and data systems you did not build.
• Strong foundation in applied statistics, experimentation, and quantitative analysis.
• Practical experience working with and evaluating modern AI/ML systems.

Responsibilities

• AI Evaluation: Develop quantitative methods and tooling to evaluate AI system quality and performance.
• Experimentation: Design and run experiments to compare approaches, understand tradeoffs, and measure the impact of changes.
• Quality Monitoring: Build repeatable processes to identify regressions, anomalies, and changes in system performance.
• Data Analysis: Extract, transform, and analyze production data to support technical and product investigations.
• Technical Investigation: Diagnose complex issues involving data, models, and production systems, identifying root causes and potential solutions.
• Measurement & Instrumentation: Partner with Engineering to improve how system quality, reliability, and performance are measured.
• Internal Tooling: Build tools and dashboards that make system performance easier to understand and monitor.
• Communication: Clearly communicate findings, tradeoffs, and recommendations to technical and non-technical stakeholders.

Skills

Python

SQL

Statistics

Experimentation

Data analysis

Communication

Education

Master's degree in CS/DS/ML

Tools

SQL databases

Cloud data platforms

Data pipelines

Job description

Syllo in the United States is seeking a Senior Data Scientist to evaluate and improve AI system quality and performance. You'll design experiments, analyze production data, and partner with Engineering, Product, and Analytics to drive measurable improvements.

This hands-on role sits at the intersection of data science and applied AI, with emphasis on instrumentation, dashboards, and clear communication of findings to technical and non-technical stakeholders.