Artificial Intelligence Engineer

Allegis Group; Hyderabad, Bengaluru; Hybrid; INR 2,500,000 - 4,000,000; Full time

Job summary

Allegis Group is seeking an AI/ML engineer to design, build, and monitor AI/LLM applications in a production-ready environment. You will develop RAG pipelines and integrations with LangChain, ensuring scalable retrieval and embedding workflows.

The role emphasizes observability, tracing, and performance metrics across multi-agent workflows, with deployment on Azure and containerized environments.

Qualifications

• Hands-on Python development experience.
• Strong experience with RAG, LangChain and LLMs.
• Experience deploying AI applications to production.
• Familiarity with Azure OpenAI and vector search.

Responsibilities

• Design, develop, and enhance AI/LLM applications using Python and LangChain.
• Build and optimize RAG pipelines for document ingestion, embeddings, vector search, and retrieval.
• Develop observability for LLMs and agents, including monitoring and tracing.
• Monitor AI metrics like latency, token usage, and retrieval accuracy.
• Create dashboards and alerts for production AI workloads on Azure.
• Collaborate with AI engineers and data stakeholders to improve reliability.

Skills

Python

RAG

LangChain

LLMs/Generative AI

Agentic AI/LangGraph

Azure OpenAI

Azure AI Search

Embeddings & semantic search

FastAPI / REST APIs

Prompt engineering

Production deployment

Tools

Docker

Kubernetes

Job description

Role & responsibilities

• Design, develop, and enhance AI/LLM applications using Python, LangChain, RAG, and Agentic AI frameworks.
• Build and optimize RAG pipelines covering document ingestion, chunking, embeddings, vector search, retrieval, reranking, and LLM response generation.
• Develop and integrate LLM/agent observability to monitor application health, performance, reliability, and AI quality.
• Implement tracing and monitoring across LLM calls, agent workflows, tools, retrieval pipelines, and APIs.
• Track key AI metrics such as latency, token usage, response quality, hallucination/error rates, retrieval accuracy, failures, and cost.
• Build dashboards, alerts, and monitoring mechanisms to identify LLM failures, degraded retrieval, latency issues, and production anomalies.
• Work with tools such as Azure Application Insights, Azure Monitor, OpenTelemetry, LangSmith, Azure AI Foundry/evaluation capabilities, or equivalent observability platforms.
• Implement LLM evaluation and RAG evaluation using appropriate metrics and feedback mechanisms.
• Monitor and troubleshoot multi-agent workflows, tool calls, prompt execution, and model responses.
• Integrate observability into FastAPI/microservices and cloud-native AI applications.
• Deploy and support AI workloads on Azure, with exposure to Docker/Kubernetes being an advantage.
• Establish logging, error handling, audit trails, and production monitoring for enterprise AI systems.
• Collaborate with AI engineers, data engineers, architects, and business stakeholders to improve AI reliability, performance, and user experience.
• Continuously identify opportunities to improve model performance, retrieval quality, scalability, latency, and operational cost.

Preferred candidate profile

Must-Have Skills

• Python strong hands-on development
• RAG – strong practical experience
• LangChain – strong hands-on experience
• LLMs / Generative AI
• Agentic AI / LangGraph
• Azure OpenAI
• Azure AI Search / Vector Search
• Embeddings, semantic search and document retrieval
• FastAPI / REST APIs
• Prompt engineering and LLM evaluation
• Production deployment of AI applications