Senior AI Agent & LLM Engineer - Production-Quality AI

Cacheflow; Mountain View (CA); Hybrid; USD 155,000 - 185,000; Full time

Job summary

Otter.ai is seeking a Senior AI Agent & LLM Engineer to build and refine AI-powered knowledge engines and chat experiences. You will own end-to-end development from ambiguous problems to production solutions, including quality measurement and cross-service improvements.

The role emphasizes hands-on backend/distributed-system work, building AI agent products, and a strong focus on evaluating and improving AI quality at scale across models, prompts, and data pipelines.

Qualifications

• 3+ years of AI Agent engineering, ML engineering, or related experience.
• Strong backend or distributed-systems engineering skills.
• Hands-on experience building and shipping AI-agent or LLM-powered products.
• Strong focus on AI quality and evaluating nondeterministic systems.
• Ability to use data, traces, logs, and qualitative examples to identify and resolve complex failures.
• Excellent cross-team collaboration across services and domains.
• High ownership, bias toward action, and capability to take problems from exploration to production.

Responsibilities

• Build AI-agent experiences for Otter AI Chat to help users reason over conversations, retrieve knowledge, and complete tasks.
• Advance Otter’s conversational knowledge engine via improved extraction, annotation, and indexing.
• Develop evaluation datasets, graders, regression tests, and release gates for AI quality.
• Diagnose failures across models, prompts, retrieval, tools, data pipelines, backend services, and workflows.
• Build shared agent infrastructure for orchestration, tracing, debugging, retries, sandboxed execution, and observability.
• Improve task completion, correctness, groundedness, reliability, latency, and cost.

Job description

Otter.ai is seeking a Senior AI Agent & LLM Engineer to build and refine AI-powered knowledge engines and chat experiences. You will own end-to-end development from ambiguous problems to production solutions, including quality measurement and cross-service improvements.

The role emphasizes hands-on backend/distributed-system work, building AI agent products, and a strong focus on evaluating and improving AI quality at scale across models, prompts, and data pipelines.