Senior AI Engineer

Mallow Technologies; Karur; On-site; INR 1,800,000 - 2,400,000; Full time

Benefits offered by this job

Competitive salary

Career growth

Cutting-edge AI projects

Collaborative culture

Job summary

Mallow Technologies is seeking a Senior AI Engineer in India to design, build, and deploy production-grade GenAI systems, including RAG pipelines, chatbots, and AI agents. You will own end-to-end solutions with a focus on reliability, observability, and cost efficiency in production.

You will work on context engineering, prompt optimization, and system monitoring, collaborating with product and engineering teams to deliver client-ready AI capabilities and guardrails.

Qualifications

• 5+ years software engineering, with 2+ years shipping LLM-based apps to production.
• Strong Python skills (async, APIs, testing) and FastAPI or equivalent.
• Hands-on experience with LLM provider APIs (OpenAI, Anthropic, Gemini, Bedrock/Azure OpenAI).
• Proven experience building RAG systems and understanding retrieval quality, chunking, context limits.
• Experience building AI agents with LangGraph or equivalent and MCP servers/clients.
• Working knowledge of vector databases and embedding models (pgvector, Qdrant, Pinecone, Weaviate).
• Experience deploying AI services on a cloud platform (AWS/GCP/Azure) with Docker and CI/CD.

Responsibilities

• Design and build end-to-end RAG pipelines, including ingestion, chunking, embeddings, hybrid search, and evaluation.
• Build conversational AI and chatbots with multi-turn memory, grounding, and human handoff flows.
• Develop AI agents and multi-agent workflows using LangGraph, LangChain, or similar frameworks.
• Implement MCP servers/clients to connect LLMs with client systems and data sources.
• Apply context engineering to optimize context window usage for accuracy and efficiency.
• Optimize prompts with versioning, A/B testing, and automated approaches.

Skills

Python

APIs (FastAPI)

LLM applications

RAG systems

LangGraph / LangChain

Cost & performance optimization

Cloud platforms (AWS/GCP/Azure)

Docker & CI/CD

Observability tooling

Tools

LangGraph

LangChain

OpenAI API

Docker

LangSmith

Langfuse

Arize Phoenix

Job description

Role Overview

We're looking for a Senior AI Engineer to design, build, and ship production-grade GenAI systems for our clients. You'll own solutions end to end: RAG systems, chatbots, and AI agents that are reliable, observable, and cost-efficient in production. This role is about applied LLM engineering, not training models from scratch.

Key Responsibilities

• Design and build RAG pipelines end to end, covering document ingestion, chunking strategies, embeddings, hybrid search, re-ranking, and retrieval evaluation
• Build conversational AI and chatbots with multi-turn memory, grounding, fallback handling, and human-handoff flows
• Develop AI agents and multi-agent workflows using frameworks such as LangGraph, LangChain, OpenAI Agents SDK, or CrewAI, including tool calling, planning, and state management
• Build and integrate MCP (Model Context Protocol) servers and clients to connect LLMs with client systems, APIs, and data sources
• Apply context engineering to decide what goes into the context window (retrieval, memory, tool outputs, summarisation, compaction) for accuracy and efficiency
• Optimise prompts systematically through versioning, A/B testing, and automated approaches (e.g., DSPy), not trial and error
• Manage tokenomics: track and reduce token usage and cost through prompt caching, model routing, batching, and right-sizing models per task
• Evaluate, optimise, and monitor AI system performance for accuracy, reliability, latency, and cost, using tracing and observability tools like LangSmith, Langfuse, or Arize Phoenix; after deployment, catch issues such as data drift, prompt or model regressions, and degraded accuracy, and ship fixes
• Build evaluation frameworks with golden datasets, LLM-as-judge, regression evals, and online/offline metrics, so that quality is measured before and after release
• Implement guardrails for prompt-injection defence, PII handling, output validation, and hallucination controls
• Collaborate with product managers, backend engineers, frontend developers, and clients to identify opportunities for AI-driven solutions, applying emerging AI technologies, frameworks, and best practices to real business problems; mentor junior AI engineers
• Fine-tune open-weight models (e.g., Llama, Mistral, Qwen, Gemma) using LoRA/QLoRA, SFT, or DPO when prompting and RAG aren't enough, and decide when fine-tuning is actually justified (an added advantage)

Required Skills & Qualifications

• 5+ years of software engineering experience, with 2+ years building and shipping LLM-based applications to production
• Strong Python skills (async, APIs, testing); FastAPI or similar
• Hands-on experience with LLM provider APIs (OpenAI, Anthropic, Gemini, Bedrock / Azure OpenAI), including tool calling and structured outputs
• Proven experience building RAG systems, and an understanding of why they fail (retrieval quality, chunking, context limits)
• Experience building AI agents with LangGraph or an equivalent framework, and hands-on experience building MCP (Model Context Protocol) servers and clients
• Working knowledge of vector databases (pgvector, Qdrant, Pinecone, Weaviate, or similar) and embedding models
• Practical experience with LLM evaluation and observability tooling
• Clear understanding of LLM cost and latency trade-offs and how to optimise them
• Experience deploying AI services on a cloud platform (AWS, GCP, or Azure) with Docker and CI/CD

Good to Have

• Hands-on fine-tuning experience (Hugging Face, Unsloth, Axolotl) and model serving (vLLM, Ollama, TGI)
• Experience with the A2A (Agent-to-Agent) protocol
• Familiarity with prompt optimisation frameworks such as DSPy
• Multimodal experience (vision + language, document parsing)
• Exposure to AI security and governance practices (red-teaming, OWASP Top 10 for LLMs)
• Client-facing delivery experience in a services or consulting setup
• Open-source contributions in the GenAI space

What We Offer

• Opportunity to work on cutting-edge AI projects with chance to shape a growing AI practice: its patterns, tooling, and standards
• A collaborative, engineering-driven culture
• Competitive compensation and growth opportunities