Data Scientist

Zywave, Inc.; United States; Remote; USD 120,000 - 160,000; Full time

Job summary

Zywave is seeking a Data Scientist to join its AI Engineering and Data Science team in the United States. The role emphasizes hands-on development, evaluation pipelines, and production ML workflows to deliver measurable business impact.

You will implement evaluation frameworks, run experiments, analyze results, and build scalable data pipelines while collaborating with cross-functional teams to ensure quality and reliability of AI systems.

Qualifications

• Strong foundation in statistics and experimental design.
• Hands-on experience with machine learning fundamentals, including model training, evaluation, and feature engineering.
• Proficiency in Python and the data science stack (pandas, scikit-learn, NumPy, matplotlib/seaborn).
• Strong SQL skills and experience working with relational databases and data warehouses.
• Experience building data pipelines or working with workflow orchestration tools.
• Familiarity with version control (Git) and collaborative software development practices.

Responsibilities

• Implement and operationalize evaluation pipelines for ML/AI models.
• Execute benchmark tests, collect performance metrics, and generate evaluation reports.
• Maintain and improve automated testing suites that assess model performance.
• Monitor evaluation results and flag performance issues or anomalies for investigation.
• Execute A/B tests and experiments with proper data collection and analysis.
• Conduct statistical analysis of experiment results, including hypothesis testing and effect size calculations.
• Create clear, actionable reports that communicate findings to stakeholders.
• Support the design of new experiments with data-driven insights.

Skills

Statistics

Experimental design

Python

SQL

Data pipelines

Data visualization

Git

Tools

Pandas

Scikit-learn

NumPy

Matplotlib

Seaborn

Job description

Data Scientist

At Zywave, we believe in building AI systems that are reliable, measurable, and continuously improving. The Data Scientist will work within our AI Engineering and Data Science team to execute and operationalize our evaluation framework for ML/AI models. This role is hands-on and execution-focused, requiring strong technical skills in statistical analysis, and production ML workflows. You'll implement evaluation pipelines, run experiments, and conduct analyses that ensure our AI systems meet quality standards and drive measurable business impact.

What you will do

Evaluation Framework Execution

• Implement and operationalize evaluation pipelines for agentic AI models based on established frameworks and methodologies.
• Execute benchmark tests, collect performance metrics, and generate evaluation reports.
• Maintain and improve automated testing suites that assess model performance across multiple dimensions.
• Monitor evaluation results and flag performance issues or anomalies for investigation.

Experimentation & A/B Testing

• Execute A/B tests and experiments according to designed protocols, ensuring proper implementation and data collection.
• Conduct statistical analysis of experiment results, including hypothesis testing, confidence intervals, and effect size calculations.
• Create clear, actionable reports that communicate experiment findings to stakeholders.
• Support the design of new experiments by providing data-driven insights and feasibility assessments.

Traditional ML & Statistical Analysis

• Build and maintain classical machine learning models to support business analytics and decision-making.
• Conduct exploratory data analysis to uncover insights and patterns in product usage and model performance data.
• Perform statistical analyses including regression analysis, time series forecasting, and cohort analysis.
• Develop predictive models for business metrics, user behavior, and model performance trends.

Data Workflows & Pipeline Management

• Build and maintain data pipelines that support evaluation workflows, analytics, and reporting.
• Ensure data quality through validation checks, monitoring, and documentation.
• Work with data engineering teams to optimize data collection and storage for evaluation needs.
• Create and maintain SQL queries, transformations, and data models.

Monitoring & Reporting

• Develop and maintain dashboards that track model performance, experiment results, and key metrics.
• Create automated reporting that provides visibility into AI system health and business impact.
• Support incident response by analyzing data to identify root causes of performance issues.
• Document processes, methodologies, and findings to build organizational knowledge.

What you should bring

Strong foundation in statistics and experimental design-you understand A/B testing, hypothesis testing, and can properly analyze experiment results.

Hands-on experience with machine learning fundamentals, including model training, evaluation, and feature engineering.

Proficiency in Python and the data science stack (pandas, scikit-learn, NumPy, matplotlib/seaborn).

Strong SQL skills and experience working with relational databases and data warehouses.

Experience building data pipelines or working with workflow orchestration tools.

Familiarity with version control (Git) and collaborative software development practices.

Strong analytical and problem-solving skills with attention to detail.

Good communication skills-you can explain technical concepts clearly and create compelling visualizations.

Self-motivated with the ability to manage multiple tasks and priorities effectively.

• (Preferred) Exposure to generative AI, LLM evaluation, or working with AI/ML products in production.
• (Preferred) Experience with data visualization tools (Tableau, Looker, or similar).