Software Engineer - 2 (Voicebot)

Exotel; Bengaluru; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Exotel is building and continuously improving its voicebot—spanning the model layer to live conversation experiences. This engineering role emphasizes producing tangible improvements in call quality and business metrics in live deployments.

You will own eval frameworks and perform fine-tuning experiments (FT, PEFT/LoRA/QLoRA) on open-weight models for domain tasks, plus benchmarking across providers and self-hosted options.

Qualifications

• Solid grounding in ANNs and transformer architecture to reason about model behavior.
• Hands-on experience with LLM evals, building or running eval harnesses and regresses.
• Hands-on experience with fine-tuning including PEFT/LoRA/QLoRA on open-weight models for real tasks.
• Working knowledge of speech/ASR‑TTS evaluation (WER, latency) across noisy/multilingual audio.
• Strong Python skills and ability to read/write production code.
• 2–4 years of software/ML engineering experience, with live production exposure.
• A track record of shipping live changes in production and measuring impact.

Responsibilities

• Build and maintain LLM/speech eval frameworks for voicebot, measuring task success and latency.
• Run fine-tuning experiments (FT, PEFT/LoRA/QLoRA) on open-weight models for domain tasks.
• Benchmark LLMs and ASR/TTS engines across providers and self-hosted options, and make data-driven recommendations.
• Diagnose and fix production conversation failures using logs, traces, and eval data.
• Ship changes into the live conversational pipeline with proper instrumentation and monitoring from day one.
• Take ownership across the SDLC: design, eval planning, deployment, and monitoring.

Skills

Transformer architectures

LLM evals

PEFT/LoRA/QLoRA

ASR-TTS evaluation

Python

Production experience

Troubleshooting in production

Tools

Open-weight models

Job description

Exotel is a leading provider of AI transformation to enterprises for customer engagement and experience. With over 20 billion annual conversations across Omni channel, voice, agents, and bots, Exotel is trusted by more than 7000 clients worldwide, spanning industries such as BFSI, Logistics, Consumer Durables, E-commerce, Healthcare, and Education.

Customer expectations are evolving, and businesses face the challenge of balancing the need for increased revenue, optimized costs, and exceptional customer experience (CX). Exotel steps forward as your transformative partner, offering an AI-powered communication solution to address all three!

The Voicebot team builds and operates Exotel's real-time voice AI product — production bots handling live phone conversations for enterprise customers.

We run real-time conversational pipelines end to end: speech recognition → LLM reasoning/orchestration → speech synthesis, with tool-calling for backend actions.

We evaluate and swap models constantly across providers, on cost, latency, and conversation quality — not vibes.

We believe in measuring what matters: a good demo isn't the same as a good eval.

The Role

You’ll be part of the team building and continuously improving Exotel's voicebot — from the model layer (fine-tuning, evals) to the live conversation experience (speech quality, latency, turn‑taking). This is an engineering role first: you build, evaluate, and ship changes that directly improve call quality and business metrics in live customer deployments.

What We Expect at This Level

Independent execution. Given a scoped problem and an agreed approach, you take it to production on your own — build, eval, deploy, monitor — without needing to be unblocked daily.

Deep ownership of eval frameworks and working knowledge of associated services/infra. You go deep on the AI side of the product — models, evals, speech quality — and know enough about the surrounding services to trace a live problem across the pipeline and see it through.

Build and maintain LLM/speech eval frameworks for the voicebot — task success, hallucination, instruction‑following, WER/latency, barge‑in and turn‑taking quality — across model and prompt changes.

Run fine‑tuning experiments (full FT, PEFT/LoRA/QLoRA) on open‑weight models for domain‑specific voicebot tasks, and produce the evidence for when fine‑tuning beats prompting.

Benchmark LLMs and ASR/TTS engines on cost, latency, and quality across providers and self‑hosted options — and make a clear recommendation from the data.

Diagnose and fix real production conversation failures — bad turn‑taking, misrecognition, latency spikes, prompt regressions — using logs, traces, and eval data, not guesswork.

Ship changes into the live conversational pipeline, with instrumentation and alerting built in from day one.

Take ownership across the SDLC for your changes: design (with a senior engineer), eval design, deployment, and monitoring.

What You Bring

Must‑have

Solid grounding in ANNs and transformer architecture — attention, tokenization, decoding strategies — enough to reason about why a model behaves a certain way, not just call an API.

Hands‑on experience with LLM evals: building or running eval harnesses, LLM‑as‑judge setups, regression suites for prompt/model changes.

Hands‑on experience with fine‑tuning, including PEFT/LoRA/QLoRA — on at least one open‑weight model, for a real task (not just a tutorial).

Working knowledge of speech/ASR‑TTS evaluation — WER, latency, diarization, common failure modes in real (noisy, accented, multilingual) audio.

Strong Python; comfortable reading/writing production code, not just notebooks.

2‑4 years of software/ML engineering experience, with at least some of it in a production system (not purely research/academic).

A track record of shipping and owning your own changes in production — you’ve been on the hook for something live.

Strong analytical rigor — you instinctively ask "how do we measure this" before shipping a change.

Good‑to‑have

Experience with real‑time audio/streaming systems and streaming vs. batch tradeoffs.

Experience with agentic orchestration and tool‑calling patterns for LLMs.

Exposure to RAG patterns — embeddings, vector stores, retrieval strategies.

Familiarity with self‑hosting/serving open‑weight models.

Familiarity with observability for AI workloads — cost tracking, quality dashboards.

Experience with multi‑tenant SaaS constraints (per‑tenant config, isolation).

Prior experience specifically in voice AI / IVR / contact‑center domains.

How We Work

You own it. Build, eval, ship, monitor — and stay on the hook when it’s running live. There’s no separate "MLOps team" to hand off to.

You measure it. No model or prompt change ships without an eval story behind it.

You collaborate. You’ll work closely with the voicebot architecture/product team and field delivery engineers shipping to enterprise customers. Good ideas win regardless of source.

You stay curious. New models and techniques land constantly — evaluating and benchmarking new options is part of the job, not a side project.

Why Exotel

Work on AI problems at real scale: live voice conversations, not offline batch jobs, for enterprise customers.

Strong senior engineers to design with, and real ownership of what you build.

A team that treats "does it actually work" as more important than "does it demo well."

Opportunity to work across the full voicebot AI stack: LLMs, speech, real‑time orchestration, and the infra it runs on.