Senior Data Scientist, Search AI Mode

Socket.dev; Mountain View (CA); On-site; USD 174,000 - 252,000; Full time

Job summary

Google Search is evolving with AI Mode, leveraging Gemini foundation models to deliver multi-modal, conversational, and synthesized intelligence to billions globally.

As a Senior Data Scientist for AI Mode, you will lead data science strategy for model evaluation, experiment design, and multi-turn user journeys, shaping product decisions with cross-functional teams.

Qualifications

• Master's degree in Statistics, Data Science, Mathematics, Physics, Economics, Operations Research, Engineering, or related quantitative field.
• 5 years of work experience using analytics to solve product or business problems, coding (e.g., Python, R, SQL), querying databases or statistical analysis, or 3 years of work experience with a PhD degree.
• Experience in experimental design, causal inference, and A/B testing.

Responsibilities

• Define the north star metrics and measurement frameworks for AI Mode (growth, quality, factuality, engagement).
• Design, analyze, and interpret large-scale live A/B experiments and user feedback for generative AI features.
• Develop statistical, NLP, and ML models to cluster query intents and analyze multi-turn behaviors.
• Serve as the primary quantitative thought partner for PMs, Engineering Leads, and Research Scientists.

Skills

Analytics problem solving

Experiment design

A/B testing

Causal inference

Education

Master's degree in quantitative field

Tools

Python

R

SQL

Pandas

NumPy

Scikit-learn

PyTorch

Job description

Minimum qualifications:

• Master's degree in Statistics, Data Science, Mathematics, Physics, Economics, Operations Research, Engineering, or a related quantitative field.
  
• 5 years of work experience using analytics to solve product or business problems, coding (e.g., Python, R, SQL), querying databases or statistical analysis, or 3 years of work experience with a PhD degree.
  
• Experience in experimental design, causal inference, and A/B testing.

Preferred qualifications:

• 8 years of work experience using analytics to solve product or business problems, coding (e.g., Python, R, SQL), querying databases or statistical analysis, or 6 years of work experience with a PhD degree.
  
• Experience with Generative AI/LLM evaluation, prompt engineering analytics, NLP, embedding-based topic modeling, or reinforcement learning feedback (RLHF/RLAIF) analysis.
• Experience working on large-scale consumer products, search engines, recommendation systems, or conversational AI interfaces.
• Strong proficiency in SQL (complex joins, analytical window functions) and scripting in Python (e.g., Pandas, NumPy, Scikit-learn, PyTorch) or R.
• Deep understanding of multi-sided ecosystem dynamics and long-term user behavior modeling.

About the job:

Google Search is undergoing a generational transformation. AI Mode (AIM) represents the next evolution of search—leveraging state-of-the-art Gemini foundation models to deliver multi-modal, conversational, and synthesized intelligence to billions of users globally.

As a Senior Data Scientist for AI Mode, you will sit at the intersection of Generative AI, Large Language Models (LLMs), and core Search. You will lead the data science strategy to evaluate model quality, design complex experimentation frameworks, understand multi-turn user journeys, and shape product strategy alongside Engineering, Product Management, Research, and UX.

In Google Search, we're reimagining what it means to search for information – any way and anywhere. To do that, we need to solve complex engineering challenges and expand our infrastructure, while maintaining a universally accessible and useful experience that people around the world rely on. In joining the Search team, you'll have an opportunity to make an impact on billions of people globally.

Individual pay is determined by factors including job-related skills, experience, and relevant education or training.

US: $174000 - $252000 (USD) + 15% bonus target + equity + benefits

Learn more about benefits at Google.

Responsibilities:

• Define the north star metrics and measurement frameworks for AI Mode (e.g., growth, answer quality, factuality/hallucination rates, task completion, user satisfaction, and long-term engagement).
  
• Design, analyze, and interpret large-scale live A/B experiments, search/interaction logs, and offline Side-by-Side (SxS) rater and user feedback for generative AI features, prompt changes, and UI paradigms.
  
• Develop statistical, NLP, and machine learning models to cluster query intents, analyze conversational/multi-turn behaviors, and measure ecosystem health (e.g., publisher web traffic and ecosystem impact).
  
• Serve as the primary quantitative thought partner for PMs, Engineering Leads, and Research Scientists, translating complex statistical analyses into high-impact product roadmaps and strategic decisions.