Lead Data Scientist, Data Products

Jobtailor; San Francisco, Northern (CA, KY); Hybrid; USD 150,000 - 210,000; Full time

Job summary

Jobtailor in San Francisco is seeking a senior data scientist to own model evaluation, measurement, and data quality across ML products. You will translate business needs into technical plans and collaborate with ML engineers and cross-functional teams.

The role requires hands-on production experience, strong SQL, and Python for data processing, with on-site presence three days per week and relocation/visa readiness.

Qualifications

• 5+ years of experience in data science or related quantitative domain.
• Hands-on ownership of model evaluation in production systems.
• Experience defining evaluation frameworks for ambiguous ML problems.
• Experience with baseline selection, offline/online metric design, and validation strategies.
• Strong SQL proficiency.
• Production-quality Python for statistical data processing.
• Fluency in metrics and measurement for consumer software products.
• Ability to translate business needs into technical plans and vice versa.
• Willingness to work on-site in the San Francisco office three days per week and relocation/visa status.

Responsibilities

• Define evaluation frameworks, offline and online metrics, and quality bars for internal models and ML products
• Build the measurement layer for cross-domain machine learning products
• Scale organizational visibility into performance across the business
• Own experimentation and monitoring for production models
• Contribute to monitoring key metrics and drift detection for enriched datasets
• Ensure quality for downstream athlete-facing experiences
• Identify and size AI/ML product opportunities
• Translate ambiguous problem spaces into scoped initiatives with defined success criteria
• Serve as the data science domain expert for the team
• Raise standards for baselines, validation, and evidence quality across ML engineers and cross-functional partners
• Partner closely with Machine Learning Engineers and cross-functional teams

Skills

Model Evaluation

SQL Proficiency

Python for Data Processing

Metrics & Measurement

Cross-functional Collaboration

Tools

Python

Pandas

Job description

• Define evaluation frameworks, offline and online metrics, and quality bars for internal models and ML products
• Build the measurement layer for cross-domain machine learning products
• Scale organizational visibility into performance across the business
• Own experimentation and monitoring for production models
• Contribute to monitoring key metrics and drift detection for enriched datasets
• Ensure quality for downstream athlete-facing experiences
• Identify and size AI/ML product opportunities
• Translate ambiguous problem spaces into scoped initiatives with defined success criteria
• Serve as the data science domain expert for the team
• Raise standards for baselines, validation, and evidence quality across ML engineers and cross-functional partners
• Partner closely with Machine Learning Engineers and cross-functional teams

Requirements

• 5+ years of experience in data science or a related quantitative domain
• Hands-on ownership of model evaluation and measurement for systems running in production
• Experience defining evaluation frameworks for ambiguous ML problems
• Experience with baseline selection, offline and online metric design, and validation strategies where ground truth is imperfect
• Strong SQL proficiency
• Comfort writing production-quality Python for statistical data processing
• Fluency in metrics and measurement for consumer software products
• Ability to translate business needs into technical plans and vice versa
• Ability to work on-site in the San Francisco office three days per week
• Must be located within commuting distance of San Francisco or willing to relocate
• Must address employment visa sponsorship status

Core Competencies

Demonstrates expertise in defining evaluation frameworks and metrics for machine learning products, with a strong focus on model evaluation, measurement, and data quality. Proficient in translating business needs into technical plans while collaborating effectively with cross-functional teams.

Highest-signal resume keywords

• Model Evaluation
• SQL Proficiency
• Python for Data Processing
• Evaluation Frameworks for ML
• Metrics and Measurement for Software

Hard Skills

• Model Evaluation
• Evaluation Frameworks
• Baseline Selection
• Metric Design
• Validation Strategies
• Statistical Data Processing
• Machine Learning
• Data Science
• Performance Monitoring
• Drift Detection

Soft Skills

• Collaboration
• Problem-Solving
• Communication

Industry Keywords

• Machine Learning Products
• Consumer Software
• Data Science Domain Expertise
• Cross-Functional Teams
• San Francisco Office