Lead AI Data Engineer

Jobtailor; Dadri; On-site; INR 4,000,000 - 8,500,000; Full time

Job summary

Jobtailor is seeking an experienced Architect to lead enterprise-grade LLM and GenAI initiatives in a dynamic setting. The role focuses on architecting agentic AI solutions, RAG pipelines, and scalable design patterns, while guiding project teams and governing best practices across multiple engagements.

The ideal candidate brings deep experience with LLMs, LangChain/LangGraph, cloud data platforms, and production-grade API development.

Qualifications

• 9–12 years total experience.
• 2–4+ years hands-on in LLM / GenAI delivery (production use cases).
• Strong hands-on experience with LLMs (Claude, OpenAI, etc.).
• RAG pipelines and retrieval optimisation.
• GPT + Agentic AI implementation experience.
• Experience with LangChain, LangGraph, or similar frameworks.
• Agent orchestration and tool-calling architectures.
• Deep understanding of LLM limitations, evaluation, and optimisation strategies.
• Strong Python/Pyspark engineering expertise (production‑grade development) with proven API integration experience.
• Deep data analysis experience and handling large volume of data.

Responsibilities

• Architect enterprise-grade agentic and LLM solutions across projects.
• Define scalable GenAI system design patterns (RAG, orchestration layers).
• Lead end-to-end delivery for RAG pipelines from data ingestion to retrieval and response generation.
• Mentor junior engineers and project teams; conduct reviews and code walkthroughs.
• Drive design reviews, governance, and best practices for GenAI initiatives.
• Build production-grade APIs/services (FastAPI, Flask) and ensure performance optimisation.

Skills

LLM Implementation Experience

GenAI

RAG Pipelines

Prompt Engineering

API Development

Data Analysis

SQL

Python

Pyspark

Agent Orchestration

Tools

LangChain

LangGraph

Azure Databricks

Snowflake

Containers

CI/CD

FastAPI

Flask

Job description

• Architect enterprise-grade agentic and LLM solutions (single-agent, multi-agent, tool-driven workflows)
• Define scalable GenAI system design patterns (RAG, orchestration layers, evaluation frameworks)
• Act as the technical anchor for GenAI initiatives across projects
• Drive design reviews, architecture governance, and best practices
• Design and build agentic systems using LLMs for use cases such as knowledge assistants, document automation & intelligence, workflow orchestration
• Implement advanced prompt engineering strategies, prompt orchestration, and reasoning chains
• Build tool-calling / function-calling frameworks for agent workflows
• Lead end-to-end implementation of RAG pipelines: Data ingestion → chunking → embeddings → vector indexing → retrieval → response generation
• Optimise retrieval quality (recall, relevance, grounding)
• Evaluate and benchmark different architectures
• Develop production-grade APIs/services (FastAPI, Flask, etc.)
• Drive code quality, testing standards, and reusable architecture components
• Ensure solutions are performance optimised (latency, cost, reliability)
• Implement LLM guardrails (hallucination control, safety filters, policy enforcement)
• Define evaluation frameworks (response quality metrics, RAG benchmarking, human-in-the-loop validation)
• Drive end-to-end delivery ownership across multiple projects
• Mentor and guide junior engineers and project teams
• Conduct technical reviews, solution walkthroughs, and code reviews.

Requirements

• 9–12 years total experience
• 2–4+ years hands‑on in LLM / GenAI delivery (production use cases)
• Strong hands‑on experience with LLMs (Claude, OpenAI, etc.)
• RAG pipelines and retrieval optimisation
• GPT + Agentic AI implementation experience
• Experience with LangChain, LangGraph, or similar frameworks
• Agent orchestration and tool-calling architectures
• Deep understanding of LLM limitations, evaluation, and optimisation strategies
• Strong Python/Pyspark engineering expertise (production‑grade development) with proven API integration experience
• Deep data analysis experience and handling large volume of data
• Fabric/Azure Databricks/Snowflake data engineering integration skills
• Good exposure to Cloud platforms (Azure/AWS/GCP)
• SQL
• Containers, CI/CD, monitoring
• Experience leading solution design or small teams
• Ability to translate business problems into AI solutions
• Strong stakeholder communication and influencing skills.

Core Competencies

Demonstrates expertise in architecting and implementing enterprise‑grade LLM solutions, with a strong focus on RAG pipelines, prompt engineering, and performance optimization. Proven ability to lead technical initiatives, mentor teams, and translate complex business challenges into effective AI solutions.

Highest‑signal resume keywords

• LLM Implementation Experience
• RAG Pipeline Development
• Python/Pyspark Engineering Expertise
• Cloud Platform Exposure (Azure/AWS/GCP)
• Data Engineering Integration Skills

Hard Skills

• LLM
• GenAI
• RAG Pipelines
• Prompt Engineering
• API Development
• Data Analysis
• SQL
• Python
• Pyspark
• Agent Orchestration

Soft Skills

• Stakeholder Communication
• Team Leadership
• Mentoring

Industry Keywords

• AI Solutions
• Document Automation
• Workflow Orchestration
• Evaluation Frameworks
• Performance Optimization

Tools & Technologies

• FastAPI
• Flask
• LangChain
• LangGraph
• Azure Databricks
• Snowflake
• Containers
• CI/CD