Evaluation Researcher: Real-Time Multimodal AI

Acceler8 Talent; San Francisco (CA); Hybrid; USD 120,000 - 180,000; Full time

Job summary

Acceler8 Talent is hiring an Evaluation Researcher for a frontier AI startup in San Francisco, onsite. You will help define, measure and improve what makes AI systems intelligent, reliable and engaging.

You’ll develop evaluation frameworks for real-time, multimodal AI systems, designing benchmarks and testing methodologies that assess speech, reasoning, interaction and conversational behaviour.

Qualifications

• Strong ML research and engineering fundamentals.
• Experience with LLM evaluations, benchmarking or model assessment.
• Strong Python and deep learning experience.
• Familiarity with multimodal models, speech AI or conversational agents.
• Ability to design experiments and translate findings into improvements.

Responsibilities

• Develop evaluation frameworks for real-time, multimodal AI systems.
• Design benchmarks and testing methodologies to measure model performance across speech, reasoning, interaction and conversational behaviour.

Skills

ML research

Python

Deep learning

LLM evaluations

Multimodal models

Experiment design

Job description

Acceler8 Talent is hiring an Evaluation Researcher for a frontier AI startup in San Francisco, onsite. You will help define, measure and improve what makes AI systems intelligent, reliable and engaging.

You’ll develop evaluation frameworks for real-time, multimodal AI systems, designing benchmarks and testing methodologies that assess speech, reasoning, interaction and conversational behaviour.