Lead Engineer - Data Engineering

Pine Labs; Dadri; On-site; INR 4,000,000 - 7,500,000; Full time

Job summary

Pine Labs is seeking a Data Engineering Leader in Noida to architect and drive real-time data pipelines, data lake management, and analytics-ready data layers. You will mentor engineers, set standards, and partner with ML, product, and business teams to unlock insights.

The role requires deep expertise in streaming systems, data models, and modern tooling, with a strong emphasis on governance and observability. Office-based role in Noida Sector 62.

Qualifications

• Proven Kafka/MSK, Debezium, and real-time event-driven architectures.
• Hands-on with Pinot, Redshift, RocksDB/NoSQL databases.
• Strong Java and Python development for custom data pipelines.
• Experience with Apache Airflow, Superset, Iceberg, Athena, Glue.

Responsibilities

• Own data engineering roadmap from streaming ingestion to analytics layers.
• Lead real-time data pipelines using Kafka/MSK, Debezium, Pinot.
• Design and maintain Java/Python data ingestion frameworks.
• Manage data lakes with Iceberg, Glue, Athena on S3.
• Define and enforce data modelling standards across Pinot, Redshift.
• Lead open-source tools and Airflow orchestration.
• Collaborate with ML, product, and business teams for analytics.
• Mentor data engineers; ensure observability, governance.

Skills

Kafka/MSK

Debezium

Real-time pipelines

Java & Python

Airflow

Superset

Iceberg

Athena/Glue

Data lake architecture

Parquet/Avro

CDC concepts

Education

Bachelors or Masters in Computer Science/Engineering

Tools

Pinot

Redshift

RocksDB/NoSQL DB

Trino/Presto

ClickHouse

Druid

dbt

Terraform/CloudFormation

GitHub Actions

Job description

We are looking for a highly skilled and motivated Data engineering leader with 5-10 years of experience to join our growing team. You will create data pipelines to move data from source to data lake, Maintain the data lake, real time transform the data to uncover actionable insights and contribute to strategic decision-making across various departments. The ideal candidate will have strong analytical and programming skills, a solid understanding of data pipeline and programming logic, and the ability to collaborate effectively with cross-functional teams. You will be helping all application teams by providing data interface, also will be ensuring that data analytics team is getting required data for their initiatives

The responsibilities we entrust you

• Own the data engineering roadmap – from streaming ingestion to analytics-ready layers.
• Lead and evolve our real-time data pipelines using Apache Kafka/MSK, Debezium CDC, and Apache Pinot.
• Design and maintain custom-built Java/Python-based frameworks for data ingestion, validation, transformation, and replication.
• Design and manage data lakes using Apache Iceberg, Glue, and Athena over S3.
• Define and enforce data modelling standards across Pinot, Redshift and warehouse layers.
• Lead implementation and scaling of open-source tools and orchestration platforms (Airflow).
• Collaborate with ML, product, and business teams to support advanced analytics and AI use cases.
• Mentor data engineers and evangelize modern architecture and engineering best practices.
• Ensure observability, lineage, data quality, and governance across the data stack.

Wha mattters in this role:

Must-Have:

• Proven expertise in Kafka/MSK, Debezium, and real-time event-driven architectures
• Hands-on experience with Pinot, Redshift, RocksDB or noSQL DB
• Strong background in custom tooling using Java and Python
• Experience with Apache Airflow, Superset, Iceberg, Athena, and Glue
• Deep understanding of data lake architecture, streaming vs batch processing, CDC concepts
• Familiarity with modern data formats (Parquet, Avro) and storage abstractions

Nice-to-Have:

• Exposure to dbt, Trino/Presto, ClickHouse, or Druid
• Familiarity with data security practices, encryption at rest/in-transit, GDPR/PCI compliance
• Experience with DevOps practices, GitHub Actions, CI/CD, and Terraform/CloudFormation

Education

• Bachelor's or Master's degree in Computer Science, Engineering, or a related field.

Communication & Ownership:

• Strong communication, interpersonal, and conflict resolutio
• n skillsAble to work both independently and as part of team

Location: Sector 62, Noida

Things you should be comfortable with:

• Working from office: 5 days a week
• Pushing the boundaries: Have a big idea? See something that you feel we should do but haven’t done? We will hustle hard to make it happen. We encourage out of the box thinking, and if you bring that with you, we will make sure you get a bag that fits all the energy you bring along.

What we value inour people:

• You take the shot: You decide fast and deliver right.
• You are the CEO of what you do: You show ownership and make things happen.
• You sing your work like an artist: You seek to learn and take pride in thework you do.