Staff AI Engineer — Agentic AI

Gnani Innovations Private Limited.; India; On-site; INR 4,000,000 - 7,500,000; Full time

Job summary

Gnani.ai is hiring a Staff AI Engineer — Agentic AI in Bengaluru to lead the intelligence layer of our platform. You will shape LLM orchestration, RAG, and multi‑agent systems while mentoring engineers and shipping production code.

This hands-on leadership role focuses on building end‑to‑end architectures, safe tool‑calling pipelines, and latency-aware workflows that support voice-first AI across 22+ Indian languages.

Qualifications

• 8+ years in software or ML engineering with 2+ years building LLM-based or agentic systems in production.
• Hands-on experience with LLM orchestration frameworks and patterns (function calling, tool use, structured outputs, streaming).
• Production RAG experience: vector databases, retrieval quality tuning, re-ranking, and eval-driven iteration.
• Experience designing multi-agent systems: task decomposition, agent coordination, memory, and failure handling.
• Strong Python; comfort with Go is a plus. Solid grasp of distributed systems (queues/messaging, Redis, Kubernetes).
• Track record of leading engineers as a tech lead or staff engineer: design reviews, mentorship, delivery ownership.

Responsibilities

• Design and own the LLM orchestration layer across hosted and self-hosted models.
• Own end-to-end RAG stack: ingestion, chunking, embedding, retrieval, re-ranking, grounding.
• Design multi-agent architecture: planner/worker patterns and inter-agent communication.
• Build tool-use and enterprise integration frameworks (CRM updates, tickets, APIs).
• Define evaluation gates and observability: traces and quality dashboards.
• Address guardrails for prompts, data privacy, and compliance in collaboration with product/legal.
• Lead and mentor a team of 4–8 engineers; roadmap planning and sprint delivery.
• Work in Bengaluru-based, in-office setup with a fast-paced shipping culture.

Skills

LLM orchestration

RAG systems

Python

Distributed systems

Leadership

Go (plus)

Multi-agent design

Tools

Redis

Kubernetes

Go

Job description

Gnani.ai builds voice-first AI for enterprises. Our products include an agentic AI platform, a voice API platform (speech APIs), and a conversation analytics platform. We build our own ASR, TTS, and small language models for 22+ Indian languages, serving BFSI, insurance, healthcare, and telecom customers at scale.

About the Role

Our agentic AI platform lets enterprises build and run autonomous voice and chat agents. As Staff AI Engineer — Agentic AI, you will own the intelligence layer of this platform: how agents think, plan, retrieve knowledge, use tools, and work together. You will set the technical direction for LLM orchestration, RAG, and multi-agent systems, and lead a team of agentic AI engineers to ship it.

This is a hands‑on leadership role. You will write code, review designs, and mentor engineers — not just manage.

What You Will Do

LLM Orchestration

• Design and own the orchestration layer that routes requests across LLMs (hosted and self-hosted SLMs), with fallbacks, caching, and cost/latency controls.
• Build prompt management, structured output handling, and tool‑calling pipelines that hold up in real‑time voice conversations (strict latency budgets).

RAG Pipelines

• Own the end‑to‑end RAG stack: ingestion, chunking, embedding, retrieval, re‑ranking, and grounding for enterprise knowledge bases.
• Improve answer accuracy and reduce hallucination for domain‑heavy verticals (BFSI, insurance, healthcare), including code‑mixed and multilingual content.
• Build freshness, versioning, and access control into retrieval so each tenant only sees its own data.

Multi‑Agent Orchestration

• Design the multi‑agent architecture: planner/worker patterns, agent hand‑offs, shared memory, and inter‑agent communication.
• Own agent memory design (contact, campaign, and agent‑level memory) and how agents learn from production feedback.

Tool Use & Enterprise Integrations

• Build the tool‑calling and integration framework that lets agents take real actions: CRM updates, ticket creation, payment flows, and API calls into customer systems.
• Make tool execution safe and auditable: schemas, validation, retries, and human‑in‑the‑loop approval where needed.

Evaluation & Observability

• Define evaluation gates: offline evals, golden test sets, persona simulators, and LLM‑as‑judge pipelines before changes ship.
• Build observability for every agent decision: traces, decision logs, and quality dashboards so failures can be found and fixed fast.

Guardrails, Safety & Compliance

• Design guardrails against prompt injection, hallucinated actions, and off‑policy behavior, with deterministic fallbacks and state recovery.
• Ensure agent behavior meets enterprise compliance needs (data privacy, consent, and disclosure rules) in partnership with product and legal teams.

Cost & Performance Engineering

• Own inference cost and latency: model selection and routing, caching, batching, and KV‑cache reuse, so agents stay fast and affordable at scale.

Team Leadership

• Lead and mentor a team of agentic AI engineers (roughly 4–8). Set direction, review designs and code, and raise the quality bar.
• Plan the agentic AI roadmap with product and platform teams. Break big goals into sprint‑sized work.
• Hire and grow the team as the platform scales.

What You Bring

Must have

• 8+ years in software or ML engineering, with 2+ years building LLM‑based or agentic systems in production.
• Deep, hands‑on experience with LLM orchestration frameworks and patterns (function calling, tool use, structured outputs, streaming) — and knowing when to skip the framework and build it yourself.
• Production RAG experience: vector databases, retrieval quality tuning, re‑ranking, and eval‑driven iteration.
• Experience designing multi‑agent systems: task decomposition, agent coordination, memory, and failure handling.
• Strong Python; comfort with Go is a plus. Solid grasp of distributed systems (queues/messaging, Redis, Kubernetes).
• Track record of leading engineers as a tech lead or staff engineer: design reviews, mentorship, delivery ownership.

Nice to have

• Real‑time or voice AI experience (latency‑sensitive pipelines, streaming ASR/TTS integration).
• Fine‑tuning or serving SLMs (vLLM, TensorRT‑LLM, or similar).
• Experience with Indic languages or code‑mixed text.
• Familiarity with enterprise compliance needs (data residency, DPDP, RBI guidelines).
• Hands on experience with Livekit and Pipecat frameworks

Why This Role

• Own a core layer of a fast‑growing agentic platform used by large enterprises, end to end.
• Work on hard, real problems: agents that talk on live phone calls with sub‑second latency budgets.
• Build on proprietary models (ASR, TTS, SLM) — not just API wrappers.
• Small, senior team. High trust, high ownership, direct access to leadership.

How We Work

Bengaluru‑based, in‑office collaboration. Sprint‑based delivery with a clear roadmap. Design docs and evals before big changes. We value engineers who ship, measure, and improve.

About Gnani.ai

Gnani.ai builds voice-first AI for enterprises. Our products include an agentic AI platform, a voice API platform (speech APIs), and a conversation analytics platform. We build our own ASR, TTS, and small language models for 22+ Indian languages, serving BFSI, insurance, healthcare, and telecom customers at scale.

Skills Required

Primary Skills Agentic RAG Systems LLM Orchestration Backend Python