Evaluations Lead, Generative Speech AI

Sanas; Palo Alto (CA); On-site; USD 140,000 - 190,000; Full time

Job summary

Sanas is building a full Speech AI suite, all working together as one platform. As Evaluations Lead, you’ll design evaluation frameworks and benchmarking systems that answer how models are evaluated at scale, sitting at the intersection of research, product, and infrastructure to build metrics, systems, and studies across all products.

This role blends scientific rigor with hands-on execution, ensuring progress is measured by real-world qualities like understanding, naturalness, and adaptability

Qualifications

• Experience designing or implementing evaluation frameworks for generative models — audio, text, or multimodal.
• Ability to turn open-ended research ideas into production-ready systems.
• Creative skills in defining quantitative metrics for inherently subjective qualities.

Responsibilities

• Identify and define the model capabilities and behaviors that matter for evaluation.
• Build and ship evaluation pipelines with robust statistical analysis and clear reporting.
• Partner directly with model training and research teams to embed evaluation into the development loop.
• Prototype new user studies and behavioral experiments that ground evaluation in real-world use.

Skills

Evaluation frameworks

Generative models

Statistical analysis

Experiment design

Production-ready system

Job description

Sanas is building a full Speech AI suite, all working together as one platform. As Evaluations Lead, you’ll design evaluation frameworks and benchmarking systems that answer how models are evaluated at scale, sitting at the intersection of research, product, and infrastructure to build metrics, systems, and studies across all products.

This role blends scientific rigor with hands-on execution, ensuring progress is measured by real-world qualities like understanding, naturalness, and adaptability