Software Engineer - 3 (Gen AI & Speech)

Exotel; Bengaluru; On-site; INR 400,000 - 700,000; Full time

Job summary

Exotel is building an AI and speech platform powering conversations for thousands of enterprise clients, including features like AI agents and conversation quality analysis. You will work across large language models, speech processing, and scalable backend systems.

This engineering role is end-to-end: design, code, deploy, operate production services, and own them across the lifecycle with reliability and performance in mind.

Qualifications

• Bachelor's or Master's degree in Computer Science or equivalent.
• 5-8 years of software engineering experience.
• Proficiency in at least one backend language; adapt quickly to new languages.
• Strong understanding of data structures, algorithms, multi-threading, and concurrency.
• Knowledge of software engineering concepts: design patterns, modularity, scalability.
• Experience with microservices and distributed systems: design, build, operate.
• Experience designing RESTful APIs and async job processing architectures.
• Production experience with databases, caching, message brokers, and search/analytics systems.
• Hands-on experience with LLM APIs (OpenAI, Gemini, Azure) and cost management.
• Experience with cloud platforms (AWS, GCP, Azure) and containers/orchestration (Docker, Kubernetes).
• Strong analytical and problem-solving skills; excellent communication; team-oriented.

Responsibilities

• Build and optimize LLM-powered pipelines for analyzing conversations at scale, managing cost and latency.
• Work on speech processing: transcription accuracy and multilingual audio challenges.
• Design and build AI agents that reason over enterprise data with guardrails.
• Develop and scale high-throughput, multi-tenant backend services for reliability and performance.
• Own cost and observability for AI workloads; model optimization and self-hosting are active areas.
• Take end-to-end ownership of the software development lifecycle from requirements to monitoring.

Skills

Backend language proficiency

Data structures

Algorithms

Multi-threading

Concurrency

Software engineering concepts

Microservices architecture

Distributed systems

RESTful APIs

Async job processing

Databases

Caching layers

Message brokers

Search/analytics systems

LLM APIs

Cloud platforms

Docker

Kubernetes

CI/CD pipelines

Analytical/problem solving

Communication skills

Team collaboration

Education

Bachelor's or Master's in Computer Science

Tools

Docker

Kubernetes

AWS

GCP

Azure

Grafana

Kibana

Elasticsearch

Job description

Exotel is a leading provider of AI transformation to enterprises for customer engagement and experience. With over 20 billion annual conversations across Omni channel, voice, agents and bots, Exotel is trusted by more than 7000 clients worldwide, spanning industries such as BFSI, Logistics, Consumer Durables, E-commerce, Healthcare and Education.

Customer expectations are evolving and businesses face the challenge of balancing the need for increased revenue, optimized costs, and exceptional customer experience (CX). Exotel steps forward as your transformative partner, offering an AI-powered communication solution to address all three!

Exotel engineering solves some really cool infrastructure level problems with the goal of ensuring no one misses a call or an SMS.

• Our focus is on building very fault-tolerant, loosely coupled, scalable and real-time distributed systems
• We emphasize a lot on clean abstractions of code, loosely coupled services and good coding practices
• We are very strong believers in "you built it, you own it!". And running a distributed system is very different from just building one!
• We are crazy about high availability

The Role

You'll be part of the team building Exotel's AI and speech platform, powering products like Conversation Quality Analysis, AI Voice Agents, and AI Chat Agents. This means working across large language models, speech processing, and distributed backend systems to process and understand millions of enterprise conversations at scale.

This is an engineering role first. You'll design, build, deploy, and operate production services, and own them end-to-end.

What You'll Do

• Build and optimize LLM-powered pipelines for analyzing conversations at scale, managing cost, latency, and quality tradeoffs across model providers.
• Work on speech processing: improving transcription accuracy, handling multilingual audio, and solving real-world audio quality challenges.
• Design and build AI agents that reason over enterprise data and deliver actionable answers with appropriate guardrails.
• Develop and scale high-throughput, multi-tenant backend services with focus on reliability and performance.
• Own cost and observability for AI workloads: usage tracking, cost attribution, and quality monitoring. Model optimization and self-hosting are active areas of investment.
• Take end-to-end ownership of the software development lifecycle: requirements, design, development, testing, deployment, and monitoring.

What You Bring

Must-have

• Bachelor's or Master's degree in Computer Science or equivalent.
• 5-8 years of software engineering experience.
• Proficiency in at least one backend language (Python, Java, Go, or similar). Ability to pick up new languages quickly; the language matters less than the engineering thinking behind it.
• Strong understanding of data structures, algorithms, multi-threading, and concurrency.
• Good understanding of software engineering concepts: design patterns, modularity, scalability.
• Experience with microservices architecture and distributed systems: designing, building, and operating them.
• Experience designing and developing RESTful APIs and async job-processing architectures.
• Production experience with databases, caching layers, message brokers, and search/analytics systems, including data modeling and scaling.
• Hands‑on experience with LLM APIs (OpenAI, Gemini, Azure, or similar): prompt engineering, structured output, cost management. If you haven't worked with LLMs yet but have strong engineering fundamentals and a willingness to learn, that works too.
• Experience with major cloud platforms (AWS, GCP, or Azure).
• Experience with containers and orchestration (Docker, Kubernetes) and CI/CD pipelines.
• Strong analytical and problem-solving skills.
• Excellent written and verbal communication skills.
• Team player, comfortable working across teams (product, data, infrastructure) in a fast-paced environment.

Good-to-have

• Understanding of RAG patterns: embeddings, vector stores, retrieval strategies.
• Comfortable with Linux, shell scripting, and developing Linux-based applications.
• Familiarity with monitoring and observability tools such as Grafana, Kibana, Elasticsearch.
• Strong networking fundamentals: DNS, load balancing, proxies, firewalls.
• Experience with ASR/TTS engines: Whisper-family models, VAD, speaker diarization, alignment, and common failure modes.
• Experience working with audio pipelines: IP streaming, format handling, noise reduction, streaming vs. batch processing.
• Experience with graph databases and graph query languages.
• Familiarity with columnar analytics stores for large-scale analytical queries.
• Experience building multi-tenant SaaS with tenant isolation and per-tenant configuration.
• Exposure to LLM observability and cost tracking tooling.
• Experience self-hosting or fine-tuning open-weight models.

How We Work

• You own it. Design, code, deploy, monitor. DevOps is a culture, not a separate team.
• You lead. Mentor and own the output of a team of 4-6 engineers. Code reviews, design reviews, and growing people are part of the job.
• You collaborate. You'll work closely with product, data, and platform teams. Good ideas win, regardless of where they come from.
• You stay curious. New models, tools, and techniques emerge constantly. We expect you to evaluate and benchmark; staying ahead is part of the role.

Why Exotel

• Work on AI problems at real scale: billions of conversations, thousands of enterprise clients.
• High autonomy and ownership. Your decisions shape the product.
• A team that values engineering depth, not just shipping fast.
• Opportunity to work across the full AI stack: LLMs, speech, agents, infrastructure.