Data Scientist, Evaluation

Apple; Seattle (WA); On-site; USD 142,000 - 263,000; Full time

Benefits offered by this job

Comprehensive medical and dental

Retirement benefits

Employee stock programs

Tuition reimbursement

Relocation assistance

Job summary

Apple is seeking a seasoned evaluation scientist to measure product quality across user-facing experiences. You will research evaluation methods for Siri and Apple Intelligence, partner with engineering teams to translate methods into technologies, and tackle data-driven problems using ML and advanced analytics.

You will work with large datasets, prototype scalable pipelines, and advocate for data-driven product improvements.

Qualifications

• Advanced degree in a quantitative field such as Statistics, OR, Bioinformatics, Economics, Psychology, CS, Sociology, Mathematics or Physics.
• 5+ years of relevant experience.
• Proven cross-functional collaboration ability.
• Experience leading projects with varying scope and depth.
• Experience addressing accuracy and variability in human annotation data.
• Proficiency in data science, ML, and analytics.
• Experience designing, conducting, analyzing experiments with emphasis on data quality and risk assessment.
• Strong programming skills in SQL and/or Spark and scripting languages like Python, R, or Scala.
• Ability to translate business questions into analytical approaches and apply statistics.

Responsibilities

• Research and develop evaluation methods to improve quality of Apple user facing products (e.g., Siri, Apple Intelligence).
• Partner with evaluation and experimentation teams to translate methods into product engineering technologies.
• Work with large datasets and tackle non-routine problems using advanced analytics, including prompt engineering and building LLMs as judges.
• Perform analysis across data collection, quality control, requirements, processing, and presentations.
• Build and prototype analysis pipelines to deliver Insights at scale.
• Develop deep knowledge of product data structures and advocate for changes to support product development.
• Collaborate with product engineering on core ML algorithms and user experience evaluations.

Skills

Data science

Machine learning

Analytics

SQL

Spark

Python

R

Scala

Education

Advanced quantitative degree

Tools

SQL

Spark

Python

R

Scala

Job description

Use evaluation science to measure product quality and inform engineering decisions across Apple’s user-facing experiences.

Responsibilities

• Research and develop evaluation methods to improve the quality of Apple user facing products, including Siri and Apple Intelligence.
• Partner with evaluation and experimentation engineering teams so methodological work is translated into technologies used in product engineering.
• Work with large, complex datasets and address difficult, non-routine analysis problems using advanced analytical methods, including prompt engineering and building LLMs as judges.
• Perform analysis spanning data collection and quality control, requirements specification, processing, and presentations.
• Build and prototype analysis pipelines iteratively to deliver insights at scale.
• Develop deep knowledge of product data structures and metrics, and advocate for changes needed to support product development.
• Collaborate with product engineering teams on core machine learning algorithms and user experience evaluations.

Requirements

• 5+ years of relevant work experience.
• Advanced degree in a quantitative field such as Statistics, Operational Research, Bioinformatics, Economics, Psychology, Computer Science, Sociology, Mathematics, Physics, or similar quantitative discipline.
• Proven ability to collaborate effectively across functions and work well in a team.
• Experience driving projects across different sizes and scopes, with an ability to decide when to go deep.
• Experience using methods that address accuracy and variability in human annotation data.
• Ability to learn new technology and skills as working requirements change.
• Proficiency in data science, machine learning, and analytics, including statistical data analysis.
• Experience crafting, conducting, analyzing, and interpreting experiments and investigations, with a focus on data quality, evaluation, and risk assessment.
• Strong programming skills, including data-querying (e.g., SQL and/or Spark) and experience with a scripting language for data processing and development (e.g., Python, R, or Scala).
• Ability to translate business questions into analytical approaches and apply statistical techniques to answer them using available data.
• Strong communication skills, including the ability to explain complex technical topics (especially causal concepts) to data scientists, engineers, and business partners.

Technologies

• SQL
• Spark
• Python
• R
• Scala
• LLMs

Benefits

• Comprehensive medical and dental coverage
• Retirement benefits
• Range of discounted products and free services
• Reimbursement for certain educational expenses, including tuition
• Discretionary employee stock programs, including eligibility for discretionary restricted stock unit awards
• Option to purchase Apple stock at a discount if voluntarily participating in Apple’s Employee Stock Purchase Plan
• May be eligible for discretionary bonuses or commission payments as well as relocation

Pay & Benefits

Base pay range for this role is $142,300 - $263,300 per year. Your base pay depends on skills, qualifications, experience, and location.

Note: Apple benefit, compensation, and employee stock programs are subject to eligibility requirements and other terms of the applicable plan or program.