Lead Data Engineer

ISA; Maharashtra; Hybrid; INR 4,000,000 - 6,500,000; Full time

Job summary

ISA in Maharashtra, India is seeking a seasoned Data Engineering Lead to own the design and evolution of the data platform, driving scalable pipelines and governance. You will mentor a team, collaborate with data science and AI teams, and align engineering strategy with business goals.

You will oversee data lakes, warehouses, and real-time streaming, define standards, ensure security, and partner with stakeholders to deliver robust data solutions that enable analytics and ML workloads.

Qualifications

• Bachelor's or Master's degree in Computer Science, Computer Engineering, Information Technology, or related field.

Responsibilities

• Define and own the data engineering roadmap, aligning pipeline, platform, and infrastructure strategy with organizational data and AI goals.
• Architect scalable, resilient data pipelines, data lakes, and warehouse solutions meeting performance, cost, and governance requirements.
• Lead end-to-end delivery of complex data engineering initiatives from requirements to production.
• Establish engineering standards, code quality practices, and data governance frameworks across the team.
• Evaluate and introduce new tools and cloud-native technologies to improve the data platform.
• Partner with Data Science and AI teams to ensure data infrastructure supports analytics and ML workloads.
• Collaborate with stakeholders to translate data requirements into robust engineering solutions.
• Lead, mentor, and develop a team of data engineers with a high-performance culture.
• Own data platform reliability, monitoring, incident response, root-cause analysis and preventive improvements.
• Provide direction on data security, access control, and compliance with governance functions.

Skills

SQL

Python

Spark

Scala

Java

Cloud platforms

Data modeling

Data governance

MLOps

CI/CD

Airflow

dbt

Kafka

Kinesis

Communication

Leadership

Education

Bachelor's or Master's degree in Computer Science, Computer Engineering, Information Technology, or related field

Tools

Apache Airflow

Prefect

dbt

Kafka

Flink

Kinesis

Cloud services

Job description

Job Purpose

Lead the Data Engineering function, owning the design and evolution of the organization's data infrastructure and platform. Drive the delivery of scalable, reliable data pipelines and systems that underpin analytics, data science, and AI capabilities, while developing team capability and aligning engineering strategy with broader business objectives.

Key Result Responsibilities

• Define and own the data engineering roadmap, aligning pipeline, platform, and infrastructure strategy with organizational data and AI goals
• Architect and oversee the design of scalable, resilient data pipelines, data lakes, and warehouse solutions that meet performance, cost, and governance requirements
• Lead end-to-end delivery of complex data engineering initiatives, from requirements through to production
• Establish and enforce engineering standards, code quality practices, and data governance frameworks across the team
• Evaluate and introduce new tools, frameworks, and cloud-native technologies to continuously improve the data platform
• Partner with Data Science and AI teams to ensure data infrastructure is fit for advanced analytics and machine learning workloads
• Collaborate with business stakeholders to translate data requirements into robust, reusable engineering solutions
• Lead, mentor, and develop a team of data engineers, supporting career growth and building a high-performance engineering culture

Key Result Responsibilities-Continued

• Own data platform reliability, monitoring, and incident response; drive root-cause resolution and preventive improvements
• Provide direction on data security, access control, and compliance in coordination with relevant governance functions

People Management

• Provides direction, coaching and guidance to team members for their career and professional development.
• Creates a conducive working environment to build and sustain a performance driven, engaged, and committed team.
• Ensures people management responsibilities are handled effectively in line with company procedures.

Qualifications (Academic, Training, Languages)

• Bachelor's or Master's degree in Computer Science, Computer Engineering, Information Technology, or a related field.
• Expert-level proficiency in SQL and Python; strong knowledge of Spark, Scala, or Java
• Deep experience with cloud data platforms (AWS, GCP, or Azure) and associated managed services
• Proven ability to design and implement complex data models across relational, NoSQL, and lakehouse architectures
• Advanced knowledge of data orchestration tools (e.g., Apache Airflow, Prefect, dbt) and CI/CD for data pipelines
• Strong understanding of data governance, data quality frameworks, and metadata management
• Familiarity with MLOps patterns and infrastructure requirements for machine learning workloads
• Ability to communicate complex technical concepts clearly to non-technical stakeholders and senior leadership
• Demonstrated leadership in building and mentoring engineering teams
• Strong commercial awareness and ability to make pragmatic engineering trade-offs aligned with business value
• Fluent in English; proficiency in Hindi is an advantage

Work Experience

• With 8-10 years of progressive experience in data engineering, with at least 2 years in a technical lead or senior engineering role.
• Demonstrated track record of delivering large-scale data infrastructure in a cloud environment. ITIL Certification is an advantage but not mandatory
• Experience with real-time and streaming data platforms (e.g., Apache Kafka, Flink, Kinesis)