Senior AI Engineer

omniXM; Pune District; On-site; INR 3,500,000 - 6,800,000; Full time

Job summary

omniXM is expanding its AI Engineering team to deepen the intelligence layer of omniSense: smarter agents, richer retrieval, better-adapted models, and tighter orchestration. You will design and implement multi-agent pipelines for survey analysis and automated reporting, enabling rapid insights from real-world feedback.

The role covers building specialized sub-agents, tool calling, HITL checkpoints, and fault-tolerant execution, with exposure to Azure, FastAPI, and RAG pipelines.

Qualifications

• 5+ years hands-on experience deploying AI/LLM systems.
• Proficient in Python and async web frameworks (FastAPI).
• Hands-on with multi-agent frameworks and RAG pipelines.
• Experience with vector stores and model fine-tuning.

Responsibilities

• Design and implement multi-agent pipelines for survey analysis and reporting.
• Build specialized sub-agents coordinating under a planner pattern.
• Implement tool calling bridging LLM reasoning with backend tools.
• Add HITL checkpoints and fault-tolerant execution for long workflows.

Skills

Python

FastAPI

Async web

Multi-agent

RAG

LLM tuning

Vector DBs

LangGraph

OpenAI API

Education

Bachelor's degree in CS or related

Tools

FAISS

ChromaDB

Qdrant

Milvus

LangChain

CrewAI

llama.cpp

vLLM

Job description

Type: Full-Time

Reporting To: Head of Engineering

ABOUT OMNIXM

omniXM builds intelligent customer experience management products. Our flagship AI product omniXMOS, includes a conversational survey analysis engine omniSense, that processes real-world business feedback at scale. The system combines deterministic data tools, LLM-powered reasoning, multi-modal report generation, and document-grounded RAG — all served through a FastAPI backend and deployed on Azure.

We are growing our AI Engineering team to deepen the intelligence layer of omniSense: smarter agents, richer retrieval, better-adapted models, and tighter orchestration.

ROLES & RESPONSIBILITIES

1. Agentic AI Design & Orchestration

• Design and implement multi-agent pipelines for survey analysis, insight generation, and automated reporting using frameworks such as LangGraph, CrewAI, or custom orchestration built on FastAPI.
• Build specialized sub-agents (e.g., a data-fetch agent, a metrics-aggregation agent, a narrative-synthesis agent, a report-formatting agent) that coordinate under a planner/orchestrator pattern.
• Implement tool calling that bridges LLM reasoning with deterministic backend tools — including ticket summaries, dimension breakdowns, order/menu extraction, and period comparisons.
• Add Human-in-the-Loop (HITL) checkpoints and agent memory (session context, cross-turn state) to support long-running, multi-step analysis workflows.
• Maintain fault-tolerant execution: retry logic, graceful degradation, and async background task handling for slow or multi-step agentic flows.

2. Retrieval-Augmented Generation (RAG)

• Own and extend the document RAG pipelines, including chunking strategies, embedding model selection, vector store management (currently SQLite-backed, with a path to Milvus/Qdrant at scale), and retrieval tuning.
• Design hybrid retrieval combining dense vector search with keyword/BM25 re-ranking to improve precision on business-domain queries.
• Build context injection pipelines that blend retrieved document chunks with live survey response data and precomputed metrics into the PromptBuilder stage.
• Evaluate retrieval quality systematically: precision@k, recall@k, faithfulness, and answer relevance against a curated benchmark question set.
• Fine-tune open-source LLMs (LLaMA 3/4, Mistral, Gemma) on omniSense-specific tasks: survey response classification, sentiment labeling, CSAT narrative generation, and structured report drafting.
• Apply LoRA / QLoRA for parameter-efficient adaptation on limited GPU budgets; evaluate Full FT selectively for core classification tasks.
• Manage training pipelines: dataset curation, formatting (JSONL instruction format), base model selection, GPU-backed training jobs, eval harness, and versioned model registry.
• Deploy fine-tuned adapters behind the existing LLMService abstraction with an inference API (vLLM / llama.cpp / Ollama), measuring latency and accuracy trade-offs vs. frontier models.

4. LLM Prompt Engineering & Evaluation

• Maintain and evolve the PromptBuilder staged prompt system — system instructions, context injection, rule sets, few-shot examples, and output format constraints.
• Build an offline eval harness: golden Q&A pairs, regression tests on known failure cases, and automated scoring (LLM-as-judge, exact match, ROUGE/BERTScore).
• Run structured A/B experiments across prompt variants, model versions, and retrieval strategies; present results and drive decisions with data.
• Extend FastAPI routers with new AI endpoints: streaming agent responses, async long-running jobs, and structured JSON outputs for frontend consumption.
• Integrate with upstream data sources (OmniServices API, Fact API, Ticket API) and ensure the AI layer handles partial failures, stale data, and schema changes gracefully.
• Instrument LLM calls with structured logging, token budgeting, cost tracking, and latency metrics (Prometheus / Azure Monitor).
• Participate in architecture reviews, propose AI capability roadmap items, and translate business requirements into well-scoped AI engineering tasks.
• Write unit and integration tests for agent pipelines, RAG retrieval, and LLM tool calls; maintain coverage in tests/unit/ .
• Document design decisions, prompt libraries, and model versioning choices so teammates can review and iterate without full context re-loading.

QUALIFICATIONS

Required

• 5+ years of hands‑on experience building and shipping AI/LLM-powered systems in production.
• Proficiency in Python and async web frameworks (FastAPI or equivalent).
• Practical experience with multi-agent frameworks: LangGraph, LangChain, CrewAI, Google ADK or equivalent; understanding of ReAct reasoning loops, tool calling, and agent state management.
• Hands‑on RAG implementation experience: embedding models (OpenAI, sentence-transformers), vector stores (FAISS, ChromaDB, Qdrant, Milvus), chunking, retrieval tuning.
• Familiarity LLM fine‑tuning: LoRA/QLoRA via Hugging Face  PEFT … etc.
• Experience working with OpenAI / Gemini / Anthropic APIs and open‑source models (LLaMA, Mistral).
• Solid understanding of prompt engineering: chain‑of‑thought, few‑shot, structured output, function/tool calling schemas.
• Experience with streaming APIs (SSE / WebSocket) for real‑time LLM output delivery.
• Familiarity with Azure or other cloud platforms for deployment and storage (Blob, App Service, AKS).

Preferred

• Experience with inference optimization: vLLM, SGLang, Triton Inference Server, llama.cpp, tensor parallelism.
• Exposure to MLOps practices: experiment tracking (MLflow, W&B), model versioning, CI/CD for ML pipelines.
• Knowledge of vector database production operations: indexing strategies, filtering, multi‑tenancy.
• Experience with Kubernetes for GPU workload scheduling and scalable LLM serving.
• Familiarity with LLM observability: LangSmith, Helicone, or custom structured logging with token/cost attribution.
• Background in survey analytics, CSAT, CX domains — a strong plus given our product focus.
• Contributions to open‑source AI projects or published technical writing.

Education

• Bachelor's degree or higher in Computer Science, Information Technology, or a related field — or equivalent demonstrated experience.