Senior Data Engineer & Generative AI Specialist

Piramal Realty; Mumbai; On-site; INR 4,000,000 - 6,400,000; Full time

Job summary

Piramal Realty seeks a senior leader to drive end-to-end design, development, and optimisation of enterprise data platforms while promoting generative AI solutions across the organisation. You will build scalable data pipelines, manage data lakes and warehouses, deploy LLM-based apps, and lead MLOps on cloud platforms.

Collaborating with analytics, product, and engineering teams, you will mentor engineers and help build a strong AI/Data capability.

Qualifications

• Bachelor's degree or equivalent with 12+ years in Data Engineering.
• 12+ years in Generative AI/LLM technologies.

Responsibilities

• Lead end-to-end design, development, and optimisation of enterprise data platforms.
• Design and implement robust data pipelines using Spark, Databricks, Kafka, Airflow.
• Develop, fine-tune, and deploy LLM-based solutions for business automation and decision support.
• Deploy and manage AI/data workloads on cloud platforms (AWS/Azure/GCP) and implement MLOps/LLMOps pipelines.
• Collaborate with analytics, business, product, and engineering teams and mentor junior engineers.

Skills

Python

SQL

PySpark

Airflow

DBT

Kubernetes

Docker

AWS

GCP

Azure

LLMs

Vector databases

RAG pipelines

Model fine-tuning

ML Ops

Education

Bachelor's degree or equivalent

Tools

Databricks

Snowflake

Kafka

Job description

Position Overview: Lead the end-to-end design, development, and optimisation of enterprise data platforms while driving the adoption of generative AI solutions across the organisation. This role combines deep expertise in modern data engineering, cloud-based architectures, and advanced AI/LLM technologies to build scalable, secure, and intelligent data-driven systems.

1. Data Engineering & Architecture

Design and implement robust data pipelines using modern tools (e.g., Spark, Databricks, Kafka, Airflow). Build and maintain scalable ETL/ELT frameworks for structured, semi-structured, and unstructured data. Own the architecture of data lakes, data warehouses (Snowflake/BigQuery/Redshift), and real-time streaming systems. Optimise data models for analytics, ML workloads, and business intelligence use cases. Ensure high data quality, governance, lineage, security, and compliance with organisational standards.

2. Generative AI & Machine Learning

Develop, fine-tune, and deploy LLM-based solutions (e.g., GPT, Llama, Claude models) for business automation, insights generation, and decision support. Build retrieval-augmented generation (RAG) architectures and vector databases (Pinecone, FAISS, Chroma). Create custom AI agents for internal workflowscredit analysis, due diligence, underwriting support, customer interactions, etc. Lead experimentation with multimodal AI (text, image, document intelligence). Collaborate with business teams to convert functional problems into scalable AI-driven solutions.

3. Platform & Infrastructure

Deploy and manage AI and data workloads on cloud platforms (AWS / Azure / GCP). Implement MLOps & LLMOps pipelines for CI/CD, automated testing, and monitoring of AI models. Integrate AI systems with enterprise applications and APIs for seamless workflow automation. Evaluate, adopt, and integrate new AI tools, frameworks, and model providers.

4. Stakeholder & Cross-Functional Collaboration

Work closely with analytics, business, product, and engineering teams to deliver impactful AI and data initiatives. Translate complex technical concepts into clear business insights. Mentor junior engineers and contribute to building an internal AI/Data capability.

Desired Qualifications & Experience

• Bachelors degree or equivalent with 612 years of experience in Data Engineering with at least 12 years in Generative AI/LLM technologies.
• Proven experience architecting scalable data platforms and deploying AI models into production.
• Background in sectors like BFSI, real estate, technology, consulting, or enterprise preferred.
• Strong command of Python, SQL, PySpark, Scala (optional).
• Expertise in data pipeline orchestration: Airflow, Prefect, DBT.
• Experience with cloud-native data architectures: AWS/GCP/Azure.
• Hands-on with LLMs, vector databases, RAG pipelines, embeddings, model fine-tuning.
• Familiarity with containerisation and orchestration (Docker, Kubernetes).
• Understanding of MLOps tools: MLflow, Vertex AI, SageMaker, Weights & Biases.