Software Engineer - 2

Exotel; Bengaluru; On-site; INR 1,800,000 - 3,000,000; Full time

Job summary

Exotel is building and operating real-time voicebot technologies, from the model layer to live conversation experiences. You will focus on improving call quality and business metrics in production deployments, owning end-to-end evaluation and experimentation.

Ideal candidates have 2–4 years in software/ML engineering, strong Python skills, and hands-on experience with LLM evals and fine-tuning (LoRA/QLoRA). This is a hands-on, ownership-driven role in Bengaluru.

Qualifications

• Solid grounding in ANNs and transformer architecture — attention, tokenization, decoding strategies.
• Hands-on experience with LLM evals: building or running eval harnesses, LLM-as-judge setups.
• Hands-on experience with fine-tuning, including PEFT/LoRA/QLoRA on open-weight models.
• Working knowledge of speech/ASR‑TTS evaluation — WER, latency, diarization, multilingual audio.

Responsibilities

• Build and maintain LLM/speech eval frameworks for the voicebot across model and prompt changes.
• Run fine-tuning experiments (FT, PEFT/LoRA/QLoRA) on open-weight models for domain-specific tasks.
• Benchmark LLMs and ASR/TTS engines on cost, latency, and quality across providers and self-hosted options.
• Diagnose and fix production conversation failures using logs, traces, and eval data.

Skills

Python

LLM evals

Fine-tuning

LoRA/QLoRA

ASR eval

Education

BS in CS/ML

Tools

Open-weight models

Python tooling

Job description

Exotel is a leading provider of AI transformation to enterprises for customer engagement and experience. With over 20 billion annual conversations across Omni channel, voice, agents, and bots, Exotel is trusted by more than 7000 clients worldwide, spanning industries such as BFSI, Logistics, Consumer Durables, E-commerce, Healthcare, and Education.

Customer expectations are evolving, and businesses face the challenge of balancing the need for increased revenue, optimized costs, and exceptional customer experience (CX). Exotel steps forward as your transformative partner, offering an AI-powered communication solution to address all three!

The Voicebot team builds and operates Exotel's real-time voice AI product — production bots handling live phone conversations for enterprise customers.

We run real-time conversational pipelines end to end: speech recognition → LLM reasoning/orchestration → speech synthesis, with tool-calling for backend actions.

We evaluate and swap models constantly across providers, on cost, latency, and conversation quality — not vibes.

We believe in measuring what matters: a good demo isn't the same as a good eval.

The Role

You’ll be part of the team building and continuously improving Exotel's voicebot — from the model layer (fine-tuning, evals) to the live conversation experience (speech quality, latency, turn-taking). This is an engineering role first: you’ll build, evaluate, and ship changes that directly improve call quality and business metrics in live customer deployments.

What We Expect at This Level

Independent execution. Given a scoped problem and an agreed approach, you take it to production on your own — build, eval, deploy, monitor — without needing to be unblocked daily.

Deep ownership of eval frameworks and working knowledge of associated services/infra. You go deep on the AI side of the product — models, evals, speech quality — and know enough about the surrounding services to trace a live problem across the pipeline and see it through.

What You’ll Do

Build and maintain LLM/speech eval frameworks for the voicebot — task success, hallucination, instruction-following, WER/latency, barge-in and turn-taking quality — across model and prompt changes.

Run fine-tuning experiments (full FT, PEFT/LoRA/QLoRA) on open-weight models for domain-specific voicebot tasks, and produce the evidence for when fine-tuning beats prompting.

Benchmark LLMs and ASR/TTS engines on cost, latency, and quality across providers and self-hosted options — and make a clear recommendation from the data.

Diagnose and fix real production conversation failures — bad turn-taking, misrecognition, latency spikes, prompt regressions — using logs, traces, and eval data, not guesswork.

Ship changes into the live conversational pipeline, with instrumentation and alerting built in from day one.

Take ownership across the SDLC for your changes: design (with a senior engineer), eval design, deployment, and monitoring.

What You Bring

Must-have

Solid grounding in ANNs and transformer architecture — attention, tokenization, decoding strategies — enough to reason about why a model behaves a certain way, not just call an API.

Hands‑on experience with LLM evals: building or running eval harnesses, LLM‑as‑judge setups, regression suites for prompt/model changes.

Hands‑on experience with fine‑tuning, including PEFT/LoRA/QLoRA — on at least one open‑weight model, for a real task (not just a tutorial).

Working knowledge of speech/ASR‑TTS evaluation — WER, latency, diarization, common failure modes in real (noisy, accented, multilingual) audio.

Strong Python; comfortable reading/writing production code, not just notebooks.

2‑4 years of software/ML engineering experience, with at least some of it in a production system (not purely research/academic).

A track record of shipping and owning your own changes in production — you’ve been on the hook for something live.

Strong analytical rigor — you instinctively ask “how do we measure this” before shipping a change.

Good-to-have

Experience with real‑time audio/streaming systems and streaming vs. batch tradeoffs.

Experience with agentic orchestration and tool‑calling patterns for LLMs.

Exposure to RAG patterns — embeddings, vector stores, retrieval strategies.

Familiarity with self‑hosting/serving open‑weight models.

Familiarity with observability for AI workloads — cost tracking, quality dashboards.

Experience with multi‑tenant SaaS constraints (per‑tenant config, isolation).

Prior experience specifically in voice AI / IVR / contact‑center domains.

How We Work

You own it. Build, eval, ship, monitor — and stay on the hook when it's running live. There's no separate “MLOps team” to hand off to.

You measure it. No model or prompt change ships without an eval story behind it.

You collaborate. You’ll work closely with the voicebot architecture/product team and field delivery engineers shipping to enterprise customers. Good ideas win regardless of source.

You stay curious. New models and techniques land constantly — evaluating and benchmarking new options is part of the job, not a side project.

Why Exotel

Work on AI problems at real scale: live voice conversations, not offline batch jobs, for enterprise customers.

Strong senior engineers to design with, and real ownership of what you build.

A team that treats “does it actually work” as more important than “does it demo well.”

Opportunity to work across the full voicebot AI stack: LLMs, speech, real‑time orchestration, and the infra it runs on.