Senior AI Agent Engineer — Scale LLM-Powered Chat

Otter.ai; Mountain View (CA); On-site; USD 185,000 - 230,000; Full time

Job summary

Otter.ai is seeking a Senior AI Agent & LLM Engineer in Mountain View, CA. You will build and improve AI systems behind Otter's conversational knowledge engine and AI Chat, spanning user-facing agent experiences, evaluation systems, and the shared platform for scalable, reliable operation.

You’ll move from ambiguous problems to production solutions, define quality metrics, and drive improvements across models, services, and user experience.

Qualifications

• 5+ years of AI Agent engineering, machine learning engineering, or related experience.
• Hands-on experience building and shipping AI-agent or LLM-powered products.
• Strong focus on AI quality and evaluation of nondeterministic systems.
• Ability to use data, traces, logs and qualitative examples to diagnose failures.
• Works effectively across services, domains, and org boundaries.

Responsibilities

• Build AI-agent experiences for Otter AI Chat to help users reason over conversations and complete tasks.
• Advance the conversational knowledge engine with better extraction, annotation, and indexing.
• Develop evaluation datasets, graders, regression tests, and release gates for AI quality.
• Diagnose failures across models, prompts, retrieval, tools, data pipelines, and backend services.
• Create shared agent infrastructure for orchestration, tracing, debugging, retries, and observability.

Skills

AI Agent engineering

LLM engineering

Model evaluation

Data tracing & debugging

Cross-team collaboration

Job description

Otter.ai is seeking a Senior AI Agent & LLM Engineer in Mountain View, CA. You will build and improve AI systems behind Otter's conversational knowledge engine and AI Chat, spanning user-facing agent experiences, evaluation systems, and the shared platform for scalable, reliable operation.

You’ll move from ambiguous problems to production solutions, define quality metrics, and drive improvements across models, services, and user experience.