ML Evaluation Scientist – Experimentation & Pipelines

Zywave, Inc.; United States; Remote; USD 120,000 - 160,000; Full time

Job summary

Zywave is seeking a Data Scientist to join its AI Engineering and Data Science team in the United States. The role emphasizes hands-on development, evaluation pipelines, and production ML workflows to deliver measurable business impact.

You will implement evaluation frameworks, run experiments, analyze results, and build scalable data pipelines while collaborating with cross-functional teams to ensure quality and reliability of AI systems.

Qualifications

• Strong foundation in statistics and experimental design.
• Hands-on experience with machine learning fundamentals, including model training, evaluation, and feature engineering.
• Proficiency in Python and the data science stack (pandas, scikit-learn, NumPy, matplotlib/seaborn).
• Strong SQL skills and experience working with relational databases and data warehouses.
• Experience building data pipelines or working with workflow orchestration tools.
• Familiarity with version control (Git) and collaborative software development practices.

Responsibilities

• Implement and operationalize evaluation pipelines for ML/AI models.
• Execute benchmark tests, collect performance metrics, and generate evaluation reports.
• Maintain and improve automated testing suites that assess model performance.
• Monitor evaluation results and flag performance issues or anomalies for investigation.
• Execute A/B tests and experiments with proper data collection and analysis.
• Conduct statistical analysis of experiment results, including hypothesis testing and effect size calculations.
• Create clear, actionable reports that communicate findings to stakeholders.
• Support the design of new experiments with data-driven insights.

Skills

Statistics

Experimental design

Python

SQL

Data pipelines

Data visualization

Git

Tools

Pandas

Scikit-learn

NumPy

Matplotlib

Seaborn

Job description

Zywave is seeking a Data Scientist to join its AI Engineering and Data Science team in the United States. The role emphasizes hands-on development, evaluation pipelines, and production ML workflows to deliver measurable business impact.

You will implement evaluation frameworks, run experiments, analyze results, and build scalable data pipelines while collaborating with cross-functional teams to ensure quality and reliability of AI systems.