Senior Applied Scientist — AI Experimentation & Analytics

Glean-Technologies; San Francisco (CA); Hybrid; USD 180,000 - 330,000; Full time

Job summary

Glean is building a world‑class Data Organization composed of product data science, applied science, data engineering and business intelligence groups. This is an applied science role based in San Francisco office.

You will collaborate with product data science and engineering teams to identify techniques, tooling and process improvements in online A/B experimentation for rigorous decision making across product domains, and develop a scalable experimentation platform.

Qualifications

• Masters degree in a quantitative field such as Statistics, Math or CS.
• PhD preferred or 3+ years in a quantitative discipline.
• Strong background in statistics or ML applied to product data.
• Proficient in Python and SQL, with experience in modern data stacks.
• Experience in B2B SaaS and A/B experimentation.
• Excellent written and verbal communication skills.

Responsibilities

• Collaborate with product data science and engineering teams to identify techniques, tooling and process improvements in online A/B experimentation to assist rigorous decision making across all relevant product domains.
• Develop and maintain our A/B experimentation platform based on stakeholder feedback
• Write code or identify vendors to deploy these techniques to production in a scalable manner that’s easy to use by engineering, product data science, product management and design teams
• Conduct end to end evaluation of various hero use cases like document/URL uploads, including evaluation set generation and criteria interpretation
• Break down end to end evaluations into more granular evaluation of various tasks & skills including content summarization/analysis/generation, multi-step reasoning & strategizing, tool selection & use, coding & system design
• Design, develop and own best practices, tools and processes across various evaluation problems, e.g. query intent classification, statistical principles for LLM stochasticity, benchmarking

Skills

Statistics

Machine Learning

Python

SQL

A/B Testing

KPIs & Dashboards

B2B SaaS

Ranking models

Education

Masters in Statistics/Math/CS

PhD in Statistics/Math/CS

Tools

dbt

Job description

Glean is building a world‑class Data Organization composed of product data science, applied science, data engineering and business intelligence groups. This is an applied science role based in San Francisco office.

You will collaborate with product data science and engineering teams to identify techniques, tooling and process improvements in online A/B experimentation for rigorous decision making across product domains, and develop a scalable experimentation platform.