Lead AI Engineer

Vahan; Karnataka; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Vahan is seeking an AI Engineer III in Bengaluru (on-site) to own outcomes across voice and WhatsApp agents, the eval suite, and the conversion funnel.

You will diagnose metric moves end-to-end, optimize latency and cost, and defend model choices with solid data.

Bring experience shipping an LLm-backed system in production and strong async Python skills; you will collaborate across data, engineering, and product.

Qualifications

• Shipped an LLM-backed system in production with real users.
• Can trace a business metric to its root cause across any layer.
• Strong async Python — you write services, not notebooks.
• Brings up evaluation before we do.
• Model-choice opinions grounded in latency and cost.
• Fluent with coding agents; can name the last time one confidently produced something wrong and how you caught it.
• Can explain a technical trade-off to a recruiter or ops lead without dumbing it down.
• Learns fast in public.

Responsibilities

• Own the outcome, not the ticket — investigate metric moves end to end and argue for the highest-leverage fix.
• Know the funnel cold: outreach, connect, qualify, handoff, interview, placement — write your own queries.
• Own agent behaviour end to end: the loop, tool surface, context management, stop conditions, retries, and timeout handling.
• Build and defend the eval suite — every behaviour change ships behind a golden set and a regression gate.
• Fight for latency and cost — time-to-first-audio and inference cost at 100K calls/day are yours to defend.
• Work the voice pipeline — streaming ASR on code-switched Hinglish, turn detection, barge-in, graceful failure on network drops.
• Debug production non-determinism — reproduce rare failures, write postmortems, close the loop with evals.
• Use coding agents hard and well — Claude Code from Day 1.

Skills

Async Python

Root-cause analysis

Explain trade-offs

Coding agents

Latency & cost optimization

Tools

Claude Code

LiveKit

Deepgram

Job description

Role title: AI Engineer III

Location: Bengaluru (On-site)

Experience: 5+ years

About Vahan
Vahan is a marketplace for blue-collar work. Zomato, Zepto, Swiggy and others hire through us; hundreds of recruitment agencies source through us. Samvaadini, our hybrid voice and WhatsApp agent, places 100,000+ outbound calls a day to people looking for delivery and warehouse work across 900+ Indian cities — qualifying them, answering questions about pay and shift timings, and handing the serious ones to a recruiter. It speaks Hindi, Hinglish, and a widening set of regional languages, mostly to people on low-end Android phones in noisy streets, many of whom have never used a chatbot before. When Samvaadini gets better, more people get placed in a job this week instead of next month.

The Role
You'll own outcomes across the system, not one component — the voice and WhatsApp agents, the eval suite, the conversion funnel they feed, and root-cause analysis across code, agent behaviour, models, and data. At this level you'll more often be handed a number that moved than a task to complete: qualified handoffs down 12%? You localize it across code, agent behaviour, a silent model change, upstream data, or a market shift, then argue for the fix with the most leverage.

What You'll Do

• Own the outcome, not the ticket — investigate metric moves end to end and argue for the highest-leverage fix (sometimes that's doing nothing)
• Know the funnel cold: outreach, connect, qualify, handoff, interview, placement — write your own queries
• Own agent behaviour end to end: the loop, tool surface, context management, stop conditions, retries, and timeout handling
• Build and defend the eval suite — every behaviour change ships behind a golden set and a regression gate
• Fight for latency and cost — time-to-first-audio and inference cost at 100K calls/day are yours to defend
• Work the voice pipeline — streaming ASR on code-switched Hinglish, turn detection, barge-in, graceful failure on network drops
• Debug production non-determinism — reproduce rare failures, write postmortems, close the loop with evals
• Use coding agents hard and well — Claude Code from Day 1

What We're Looking For

• Shipped an LLM-backed system in production, with real users, long enough for it to break
• Can trace a business metric to its root cause across any layer — as comfortable in a warehouse query as a stack trace
• Strong async Python — you write services, not notebooks
• Brings up evaluation before we do
• Model-choice opinions grounded in latency and cost — and you change them when shown better numbers
• Fluent with coding agents; can name the last time one confidently produced something wrong and how you caught it
• Can explain a technical trade-off to a recruiter or ops lead without dumbing it down
• Learns fast in public

Nice to Have (Genuinely Not Required)
Real-time voice stacks (LiveKit, Pipecat, Deepgram, ElevenLabs, Cartesia, telephony); Hindi/Hinglish or code-switched NLP; fine-tuning/distillation; durable workflow orchestration; ClickHouse or comparable analytical SQL; open-source work or public writing; MCP server authoring.