AI Evaluation Platform Tech Lead

Candidate; Mountain View (CA); Hybrid; USD 250,000 - 350,000; Full time

Job summary

Glean, Mountain View-based, is seeking a Tech Lead Manager of AI Evaluations to shape how we evaluate, test, and ship AI features across products. You will lead a team of applied AI scientists and engineers to build scalable evaluation infrastructure and continuous feedback loops that improve models, agents, and workflows.

You will translate research into reliable evaluation systems, define rubrics and signals, and work with R&D, product, and marketing to impact customer outcomes.

Qualifications

• 10+ years in software engineering, ML infra, or data systems with production systems.
• 5+ years in engineering or AI team management with cross-functional delivery.
• Hands-on in evaluating LLM/GenAI products and autograders or graders.
• Familiarity with benchmarks, datasets, and deterministic vs human judgments.
• Strong Python, APIs, testing, and distributed systems fundamentals.
• Experience with cloud infra, Kubernetes, data stores, and observability.

Responsibilities

• Own the AI evaluation eng roadmap and set end-to-end technical direction.
• Build scalable evaluation infra for datasets, graders, trace capture, and dashboards.
• Define representative evaluation sets and protected hold-out signals.
• Define a quality measurement system covering correctness, relevance, and task success.
• Develop deterministic checks, model-based graders, and human evaluation rubrics.
• Integrate evaluation into CI, pre-release suites, and production feedback loops.
• Bridge R&D, product, and marketing to turn results into product improvements.
• Lead and coach a high-performing team of applied AI scientists and engineers.

Skills

Python

ML infra

Evaluation systems

Leadership

Cross-team collaboration

CI/CD

Kubernetes

Education

MS/PhD in Computer Science, AI or related field

Job description

Glean, Mountain View-based, is seeking a Tech Lead Manager of AI Evaluations to shape how we evaluate, test, and ship AI features across products. You will lead a team of applied AI scientists and engineers to build scalable evaluation infrastructure and continuous feedback loops that improve models, agents, and workflows.

You will translate research into reliable evaluation systems, define rubrics and signals, and work with R&D, product, and marketing to impact customer outcomes.