Senior/Staff Applied Scientist

The Rundown AI, Inc.; Palo Alto (CA); On-site; USD 175,000 - 230,000; Full time

Job summary

An innovative company is seeking a talented applied scientist to join their data organization. In this role, you will leverage your expertise in statistics and machine learning to enhance product performance and improve decision-making through data-driven insights. You will work closely with cross-functional teams to build and maintain A/B testing platforms and evaluate generative AI use cases. This is an exciting opportunity to contribute to cutting-edge AI technology in a flexible and inclusive work environment, where your skills will directly impact the productivity of data teams and enhance overall company performance.

Benefits

Medical, Vision and Dental coverage

Flexible work environment

401k

Company events

Home office improvement stipend

Annual education stipend

Wellness stipend

Healthy lunches and dinners provided daily

Qualifications

• 5+ years of experience in a quantitative field with a focus on practical applications.
• Strong proficiency in Python and SQL for data management and analysis.

Responsibilities

• Define KPIs and create data visualizations for document indexing and retrieval.
• Collaborate on A/B experimentation and develop scalable platforms.

Skills

Statistics

Machine Learning

Python

SQL

Data Analysis

Education

Masters in Statistics

PhD in Mathematics

PhD in Computer Science

Tools

dbt

A/B Testing Platforms

Job description

About Glean

We’re on a mission to make knowledge work faster and more humane. We believe that AI will fundamentally transform how people work. In the future, everyone will work in tandem with expert AI assistants who find knowledge, create and synthesize information, and execute work. These assistants will free people up to focus on the higher-level, creative aspects of their work.

We’re building a system of intelligence for every company in the world. On the surface, you can think of it as Google + ChatGPT for the enterprise. Under the hood, our platform is the connective tissue between AI and knowledge. It brings all of a company’s knowledge together, understands it at a deep level, provides industry-leading search relevance over it, and connects it to generative AI agents and applications.

Glean was founded by a seasoned team of former Google search and Facebook engineers who saw a need in the enterprise space for their technical depth and passion for AI. We’re a diverse team of curious and creative people who want to help each other get big things done—so we can help other teams do the same.

We're backed by some of the Valley's leading venture capitalists—including Sequoia, Kleiner Perkins, Lightspeed, and General Catalyst—and have assembled a world-class team with senior leadership experience at Google, Slack, Facebook, Dropbox, LinkedIn, Stripe, Rubrik, Uber, Intercom, Pinterest, Palantir, and others.

Role

Glean is building a world-class Data Organization composed of product data science, applied science, data engineering and business intelligence groups. This is an applied science role based in our Palo Alto headquarters. It covers a subset of:

Ranking & RAG:

• Define KPIs, create data foundations and build visualizations to measure the efficacy of Glean’s world-leading document indexing, retrieval, ranking infrastructure, which powers its search and LLM-powered products, as well as other ways of content personalization such as ranking agents in an agent library.
• Conduct rigorous empirical analyses rooted in sophisticated techniques and a deep understanding of our ranking stack to identify & prototype ways to improve ranking at Glean. ML ENG teams would iterate & deploy these techniques into production.

A/B Experimentation:

• Collaborate with product data science and engineering teams to identify techniques, tooling and process improvements in online A/B experimentation to assist rigorous decision making across all relevant product domains.
• Develop and maintain our A/B experimentation platform based on stakeholder feedback
• Write code or identify vendors to deploy these techniques to production in a scalable manner that’s easy to use by engineering, product data science, product management and design teams

Generative AI evaluation:

• End to end evaluation of various hero use cases like document/URL uploads, including evaluation set generation, coming up with evaluation criteria and methods to interpret the results.
• Breaking down end to end evaluations into more granular evaluation of various tasks & skills including but not limited to content summarization/analysis/generation, multi-step reasoning & strategizing, tool selection & use, coding & system design.
• Design, development and ownership of best practices, tools and processes across various evaluation problems, e.g. query intent classification, standardizing the use of best statistical principles to handle LLM stochasticity, industry benchmarking.

You will thrive at this role if:

You have a Masters/PhD degree in Statistics, Mathematics or Computer Science, or another quantitative field.

• You have 5+ years of experience as a Masters degree holder, 3+ as a PhD degree holders
• You’re strong in statistics and/or machine learning. You have experience in applying these skills into tangible improvements in products, internal tools and processes in a pragmatic way that puts business urgencies first.
• You are very proficient in Python, e.g. proficient enough to maintain an internal source-controlled library used by dozens of others.
• You are concise and precise in written and verbal communication. Technical documentation is your strong suit.
• You are proficient in SQL and the modern data stack (e.g. source-controlled dbt pipelines for ETL/ELT).
• You are strong at defining good product KPIs/guardrail metrics, dashboarding and analysis of raw data to derive strategic insights.

You are a particularly good fit if:

• You have experience in B2B SaaS.
• You have experience working on ranking, developing and maintaining A/B experimentation platforms and/or ML measurement problems.
• You are passionate about using AI to improve the productivity of data teams as well as non-data professionals trying to derive more value of their company’s data.

Benefits

• Competitive compensation
• Medical, Vision and Dental coverage
• Flexible work environment and time-off policy
• 401k
• Company events
• A home office improvement stipend when you first join
• Annual education stipend
• Wellness stipend
• Healthy lunches and dinners provided daily

For California based applicants:

The standard base salary range for this position is $175,000 - 230,000 annually. Compensation offered will be determined by factors such as location, level, job-related knowledge, skills, and experience. Certain roles may be eligible for variable compensation, equity, and benefits.

We are a diverse bunch of people and we want to continue to attract and retain a diverse range of people into our organization. We're committed to an inclusive and diverse company. We do not discriminate based on gender, ethnicity, sexual orientation, religion, civil or family status, age, disability, or race.