Staff AI Engineer — Agentic AI

gnani.ai; Bengaluru; On-site; INR 6,000,000 - 9,000,000; Full time

Job summary

gnani.ai is seeking a Staff AI Engineer to own the intelligence layer of its agentic platform from Bengaluru. This hands-on leadership role requires coding, design reviews, and mentorship while shipping scalable LLM-based systems.

You will steer LLM orchestration, RAG, and multi-agent architecture; lead a small senior team; and align product and technical roadmaps with enterprise needs. Office-based in Bengaluru with sprint-driven delivery.

Qualifications

• 8+ years in software or ML engineering, with 2+ years building LLM-based or agentic systems in production.
• Deep, hands-on experience with LLM orchestration frameworks and patterns (function calling, tool use, structured outputs, streaming).
• Production RAG experience: vector databases, retrieval quality tuning, re-ranking, and eval-driven iteration.
• Experience designing multi-agent systems: task decomposition, agent coordination, memory, and failure handling.
• Strong Python; comfort with Go is a plus. Solid grasp of distributed systems (queues/messaging, Redis, Kubernetes).
• Track record of leading engineers as a tech lead or staff engineer: design reviews, mentorship, delivery ownership.

Responsibilities

• Design and own the orchestration layer that routes requests across LLMs, with fallbacks, caching, and cost/latency controls.
• Build prompt management, structured output handling, and tool-calling pipelines for real-time voice conversations.
• Own the end-to-end RAG stack: ingestion, chunking, embedding, retrieval, re-ranking, grounding.
• Improve answer accuracy and reduce hallucination for domain-heavy verticals.
• Build freshness, versioning, and access control into retrieval.
• Design the multi-agent architecture: planner/worker patterns, agent hand-offs, shared memory, and inter-agent communication.
• Own agent memory design and how agents learn from production feedback.
• Build the tool-calling and integration framework for CRM updates, ticket creation, payments, and API calls.
• Make tool execution safe and auditable: schemas, validation, retries, and human-in-the-loop approval where needed.
• Define evaluation gates: offline evals, golden test sets, persona simulators, and LLM-as-judge pipelines.
• Build observability for agent decisions: traces, decision logs, and quality dashboards.
• Design guardrails against prompt injection, hallucinated actions, and off-policy behavior; ensure compliance with data privacy rules.
• Own inference cost and latency: model selection, routing, caching, batching; mentor a team of 4–8 engineers.

Skills

8+ years experience

LLM orchestration

RAG experience

Multi-agent design

Python

Go

Distributed systems

Technical leadership

Tools

Redis

Kubernetes

TensorRT-LLM

Job description

About the Role:

Our agentic AI platform lets enterprises build and run autonomous voice and chat agents. As Staff AI Engineer — Agentic AI, you will own the intelligence layer of this platform: how agents think, plan, retrieve knowledge, use tools, and work together. You will set the technical direction for LLM orchestration, RAG, and multi-agent systems, and lead a team of agentic AI engineers to ship it.

This is a hands-on leadership role. You will write code, review designs, and mentor engineers — not just manage.

What You Will Do:

LLM Orchestration

• Design and own the orchestration layer that routes requests across LLMs (hosted and self-hosted SLMs), with fallbacks, caching, and cost/latency controls.
• Build prompt management, structured output handling, and tool-calling pipelines that hold up in real-time voice conversations (strict latency budgets).
• Own the end-to-end RAG stack: ingestion, chunking, embedding, retrieval, re-ranking, and grounding for enterprise knowledge bases.
• Improve answer accuracy and reduce hallucination for domain-heavy verticals (BFSI, insurance, healthcare), including code-mixed and multilingual content.
• Build freshness, versioning, and access control into retrieval so each tenant only sees its own data.

Multi-Agent Orchestration

• Design the multi-agent architecture: planner/worker patterns, agent hand-offs, shared memory, and inter-agent communication.
• Own agent memory design (contact, campaign, and agent-level memory) and how agents learn from production feedback.

Tool Use & Enterprise Integrations

• Build the tool-calling and integration framework that lets agents take real actions: CRM updates, ticket creation, payment flows, and API calls into customer systems.
• Make tool execution safe and auditable: schemas, validation, retries, and human-in-the-loop approval where needed.

Evaluation & Observability

• Define evaluation gates: offline evals, golden test sets, persona simulators, and LLM-as-judge pipelines before changes ship.
• Build observability for every agent decision: traces, decision logs, and quality dashboards so failures can be found and fixed fast.

Guardrails, Safety & Compliance

• Design guardrails against prompt injection, hallucinated actions, and off-policy behavior, with deterministic fallbacks and state recovery.
• Ensure agent behavior meets enterprise compliance needs (data privacy, consent, and disclosure rules) in partnership with product and legal teams.

Cost & Performance Engineering

• Own inference cost and latency: model selection and routing, caching, batching, and KV-cache reuse, so agents stay fast and affordable at scale.
• Lead and mentor a team of agentic AI engineers (roughly 4–8). Set direction, review designs and code, and raise the quality bar.
• Plan the agentic AI roadmap with product and platform teams. Break big goals into sprint-sized work.
• Hire and grow the team as the platform scales.

What You Bring:

Must have

• 8+ years in software or ML engineering, with 2+ years building LLM-based or agentic systems in production.
• Deep, hands‑on experience with LLM orchestration frameworks and patterns (function calling, tool use, structured outputs, streaming) — and knowing when to skip the framework and build it yourself.
• Production RAG experience: vector databases, retrieval quality tuning, re‑ranking, and eval‑driven iteration.
• Experience designing multi‑agent systems: task decomposition, agent coordination, memory, and failure handling.
• Strong Python; comfort with Go is a plus. Solid grasp of distributed systems (queues/messaging, Redis, Kubernetes).
• Track record of leading engineers as a tech lead or staff engineer: design reviews, mentorship, delivery ownership.

Nice to have

• Real‑time or voice AI experience (latency‑sensitive pipelines, streaming ASR/TTS integration).
• Fine‑tuning or serving SLMs (vLLM, TensorRT-LLM, or similar).
• Experience with Indic languages or code‑mixed text.
• Familiarity with enterprise compliance needs (data residency, DPDP, RBI guidelines).
• Hands on experience with Livekit and Pipecat frameworks

Why This Role

• Own a core layer of a fast‑growing agentic platform used by large enterprises, end to end.
• Work on hard, real problems: agents that talk on live phone calls with sub‑second latency budgets.
• Build on proprietary models (ASR, TTS, SLM) — not just API wrappers.
• Small, senior team. High trust, high ownership, direct access to leadership.

How We Work

Bengaluru‑based, in‑office collaboration. Sprint‑based delivery with a clear roadmap. Design docs and evals before big changes. We value engineers who ship, measure, and improve.