Staff AI Engineer — Agentic AI

gnani.ai; Bengaluru; On-site; INR 4,000,000 - 12,000,000; Full time

Job summary

gnani.ai seeks a Staff AI Engineer — Agentic AI in Bengaluru to own the intelligence layer. You will lead LLM orchestration, retrieval-augmented generation, and multi-agent coordination, building safe, scalable systems for enterprise conversations.

You will mentor engineers, review designs, and ship hands-on engineering work. In-office, fast-paced, with strong ownership and cross-functional collaboration.

Qualifications

• 8+ years in software or ML engineering.
• 2+ years building LLM-based or agentic systems in production.
• Production RAG experience: vector databases, retrieval quality tuning, re-ranking.
• Experience designing multi-agent systems: task decomposition, memory, coordination.
• Strong Python; comfort with Go is a plus.
• Lead engineers as tech lead or staff: reviews, mentorship, ownership.

Responsibilities

• Design and own LLM orchestration layer across models and tools.
• Architect multi-agent systems with planner/worker patterns and memory.
• Build tool-calling and enterprise integrations with safe, auditable flows.
• Define offline evaluations, gold tests, and observability dashboards.
• Design guardrails for prompt injection and data privacy with compliance in mind.
• Own cost and latency: model routing, caching, batching, KV-cache reuse.
• Lead and mentor a team of 4–8 engineers and set roadmap.

Skills

LLM orchestration

Python

Distributed systems

Leadership

Go

Job description

About the Role

Our agentic AI platform lets enterprises build and run autonomous voice and chat agents. As Staff AI Engineer — Agentic AI, you will own the intelligence layer of this platform: how agents think, plan, retrieve knowledge, use tools, and work together. You will set the technical direction for LLM orchestration, RAG, and multi-agent systems, and lead a team of agentic AI engineers to ship it.

This is a hands-on leadership role. You will write code, review designs, and mentor engineers — not just manage.

What You Will Do

LLM Orchestration

• Design and own the orchestration layer that routes requests across LLMs (hosted and self-hosted SLMs), with fallbacks, caching, and cost/latency controls.
• Build prompt management, structured output handling, and tool-calling pipelines that hold up in real-time voice conversations (strict latency budgets).
• Own the end-to-end RAG stack: ingestion, chunking, embedding, retrieval, re-ranking, and grounding for enterprise knowledge bases.
• Improve answer accuracy and reduce hallucination for domain-heavy verticals (BFSI, insurance, healthcare), including code-mixed and multilingual content.
• Build freshness, versioning, and access control into retrieval so each tenant only sees its own data.

Multi-Agent Orchestration

• Design the multi-agent architecture: planner/worker patterns, agent hand-offs, shared memory, and inter-agent communication.
• Own agent memory design (contact, campaign, and agent-level memory) and how agents learn from production feedback.

Tool Use & Enterprise Integrations

• Build the tool-calling and integration framework that lets agents take real actions: CRM updates, ticket creation, payment flows, and API calls into customer systems.
• Make tool execution safe and auditable: schemas, validation, retries, and human-in-the-loop approval where needed.

Evaluation & Observability

• Define evaluation gates: offline evals, golden test sets, persona simulators, and LLM-as-judge pipelines before changes ship.
• Build observability for every agent decision: traces, decision logs, and quality dashboards so failures can be found and fixed fast.

Guardrails, Safety & Compliance

• Design guardrails against prompt injection, hallucinated actions, and off-policy behavior, with deterministic fallbacks and state recovery.
• Ensure agent behavior meets enterprise compliance needs (data privacy, consent, and disclosure rules) in partnership with product and legal teams.

Cost & Performance Engineering

• Own inference cost and latency: model selection and routing, caching, batching, and KV-cache reuse, so agents stay fast and affordable at scale.
• Lead and mentor a team of agentic AI engineers (roughly 4–8). Set direction, review designs and code, and raise the quality bar.
• Plan the agentic AI roadmap with product and platform teams. Break big goals into sprint-sized work.
• Hire and grow the team as the platform scales.

What You Bring

Must have

• 8+ years in software or ML engineering, with 2+ years building LLM-based or agentic systems in production.
• Deep, hands-on experience with LLM orchestration frameworks and patterns (function calling, tool use, structured outputs, streaming) — and knowing when to skip the framework and build it yourself.
• Production RAG experience: vector databases, retrieval quality tuning, re-ranking, and eval-driven iteration.
• Experience designing multi-agent systems: task decomposition, agent coordination, memory, and failure handling.
• Strong Python; comfort with Go is a plus. Solid grasp of distributed systems (queues/messaging, Redis, Kubernetes).
• Track record of leading engineers as a tech lead or staff engineer: design reviews, mentorship, delivery ownership.

Nice to have

• Real-time or voice AI experience (latency-sensitive pipelines, streaming ASR/TTS integration).
• Fine-tuning or serving SLMs (vLLM, TensorRT-LLM, or similar).
• Experience with Indic languages or code-mixed text.
• Familiarity with enterprise compliance needs (data residency, DPDP, RBI guidelines).
• Hands on experience with Livekit and Pipecat frameworks

Why This Role

• Own a core layer of a fast-growing agentic platform used by large enterprises, end to end.
• Work on hard, real problems: agents that talk on live phone calls with sub-second latency budgets.
• Build on proprietary models (ASR, TTS, SLM) — not just API wrappers.
• Small, senior team. High trust, high ownership, direct access to leadership.

How We Work

Bengaluru-based, in-office collaboration. Sprint-based delivery with a clear roadmap. Design docs and evals before big changes. We value engineers who ship, measure, and improve.