Senior Data Engineer (AI Platforms)

Confidential Client; Bengaluru; Presencial; INR 4.000.000 - 7.000.000; Jornada completa

Descripción de la vacante

Confidential Client is seeking a Senior Data Engineer specializing in AI Platforms to lead enterprise-scale data initiatives. You will architect scalable data platforms, design feature stores, and optimize pipelines for AI/ML workloads across cloud ecosystems.

The role requires 8–12 years in data engineering, leadership experience, and collaboration with Data Science, MLOps, and Platform teams to enforce governance and security standards.

Formación

• 8–12 years of enterprise-scale data engineering experience.
• Strong cloud-native data architectures experience.
• Experience with feature stores and AI data platforms.
• Leading large-scale data platform initiatives and mentoring teams.
• Experience with data governance, security, and compliance.

Responsabilidades

• Architect and build enterprise-scale AI data platforms.
• Lead development of scalable batch and real-time pipelines.
• Design cloud-native lakehouse architectures.
• Optimize infrastructure for AI/ML workloads.
• Collaborate with Data Science, MLOps, and Platform teams.
• Establish data governance, lineage, and security standards.
• Implement orchestration, monitoring, and automation for workflows.
• Drive performance tuning and cost optimization.
• Mentor junior engineers and codify best practices.
• Support AI/GenAI data enablement initiatives.

Conocimientos

Python

SQL

PySpark

Spark

Databricks

Kafka

Airflow

AWS

Azure

GCP

Feature Stores

Lakehouse

CI/CD for data platforms

MLOps integration

Leadership

Stakeholder management

Herramientas

Hadoop

Terraform

Docker

Kubernetes

Delta Lake

Data Catalogs

Data mesh concepts

Descripción del empleo

Senior Data Engineer (AI Platforms)

We are seeking an experienced Senior Data Engineer specializing in AI Platforms with 8–12 years of expertise in enterprise-scale data engineering, AI infrastructure, and advanced analytics platforms. The ideal candidate should have strong experience designing scalable cloud-native data architectures, feature stores, and high-performance pipelines that support AI/ML ecosystems. The candidate will lead large-scale data platform initiatives, mentor teams, and drive data engineering best practices across the organization.

Key Responsibilities

• Architect and build enterprise-scale AI data platforms and feature engineering frameworks.
• Lead development of scalable batch and real-time data pipelines.
• Design cloud-native data lakehouse architectures.
• Optimize data infrastructure for AI/ML workloads and advanced analytics.
• Collaborate with Data Science, MLOps, and Platform Engineering teams.
• Establish data governance, lineage, security, and compliance standards.
• Implement orchestration, monitoring, and automation for data workflows.
• Drive performance tuning and cost optimization initiatives.
• Mentor junior engineers and define engineering best practices.
• Support AI/GenAI data enablement initiatives.

Requirements

• Mandatory Skills:
• Strong expertise in Python, SQL, PySpark, Spark.
• Experience with Databricks, Kafka, Airflow, and Hadoop ecosystem.
• Strong cloud expertise in AWS, Azure, or GCP.
• Experience designing Feature Stores, Lakehouse architectures, and Distributed data systems.
• Knowledge of Delta Lake, Data governance, Data security, and CI/CD for data platforms.
• Strong experience in Real-time streaming and Data pipeline optimization.
• Leadership and stakeholder management experience.
• Preferred Skills:
• Experience with Snowflake, BigQuery, Redshift, and Vector Databases.
• Knowledge of Generative AI data infrastructure, LLM data pipelines, and MLOps integration.
• Familiarity with Kubernetes, Terraform, Docker, and DevOps practices.
• Exposure to Data Mesh architecture and Data Fabric concepts.
• Experience in regulated industries like BFSI or Healthcare preferred.

Work Arrangements

• Type: Full-Time
• Location: Bengaluru, Karnataka, India
• Remote: Not specified