Sr.Engineer/Lead- Data Engineering

IndiaMART

Noida

On-site

INR 2,400,000 - 4,200,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

IndiaMART in Noida seeks an experienced Data Engineering Lead with at least 3 years of hands-on Spark experience. You will design, develop, and maintain our data infrastructure and lead a team of data engineers to deliver scalable, high-quality data solutions.

You will mentor engineers, collaborate with data scientists, analysts, and software engineers, and drive ELT/ETL pipelines, data modeling, and data warehousing leveraging cloud services and modern data architectures.

Qualifications

  • Minimum 3 years of hands-on data engineering experience.
  • Strong Spark expertise and big data pipeline development.
  • Experience leading a data engineering team and mentoring engineers.
  • Proficient in Python and/or Java for data processing tasks.
  • Knowledge of data warehousing, data modeling, and distributed systems.
  • Experience with cloud data services (AWS/GCP/Azure).

Responsibilities

  • Lead a team of data engineers in designing, building, and maintaining robust data pipelines using Spark.
  • Collaborate with data scientists, analysts, and software engineers to define data requirements.
  • Architect and optimize data storage and processing solutions for performance and scalability.
  • Design and implement data quality and validation processes.
  • Stay updated with industry trends and best practices in data engineering.
  • Mentor team members through code reviews and knowledge sharing.
  • Participate in full development lifecycle from requirements to deployment and monitoring.
  • Troubleshoot and resolve pipeline issues and bottlenecks.
  • Maintain documentation for data pipelines, processes, and procedures.

Skills

Apache Spark
Python
Java
ELT/ETL pipelines
Distributed computing
Cloud platforms (AWS/GCP/Azure)
Data modeling

Education

B.E./B.Tech in CS/IT

Tools

Apache Spark
EMR
DataProc
BigQuery
Kafka
Airflow
Delta Lake / Iceberg
Kubernetes
Git

Job description

Job Summary:

We are seeking an experienced and motivated Data Engineering Lead to join our growing team. The ideal candidate will have a minimum of 3 years of hands‑on experience in data engineering, with a strong focus on Spark. As a Data Engineering Lead, you will play a pivotal role in designing, developing, and maintaining our data infrastructure, as well as leading a team of data engineers to deliver high‑quality solutions.

Roles & Responsibilities:
  • Lead a team of data engineers in designing, building, and maintaining robust and scalable data pipelines and ELT/ ETL processes using Spark.
  • Collaborate closely with cross‑functional teams including data scientists, analysts, and software engineers to understand data requirements and deliver effective solutions.
  • Architect and optimize data storage and processing solutions for performance, reliability, and scalability.
  • Design and implement data quality and data validation processes to ensure the accuracy and integrity of our data.
  • Stay up-to-date with the latest industry trends and best practices in data engineering and big data technologies.
  • Mentor and guide team members, providing technical leadership, code reviews, and knowledge sharing.
  • Participate in the full development lifecycle, from requirements gathering to deployment and monitoring.
  • Troubleshoot and resolve data pipeline issues, performance bottlenecks, and other technical challenges.
  • Develop and maintain documentation for data pipelines, processes, and procedures.
Skill Required:
  • Strong expertise in Apache Spark for processing and analyzing large datasets.
  • Experience of working with EMR and DataProc is a plus.
  • Strong coding experinence in programming languages- Python, or Java.
  • Solid understanding of distributed computing concepts and data partitioning techniques.
  • Experience with cloud platforms such as AWS, GCP, or Azure and their data services.
  • Proven track record of designing and implementing ELT/ ETL processes and data pipelines.
  • Experience with data modeling, data warehousing using BigQuery, and data architecture principles.
  • Experience or Familiarity with LakeHouse architecture with Apache delta files and iceberg.
  • Experience with Apache Kafka, Apache Airflow, Debezium and Kafka Connect on scale.
  • Experience with zero code or low code data pipelines using open source.
  • Experience of deploying data pipelines using Containers and Kubernetes.
  • Excellent problem‑solving skills and the ability to work in a fast‑paced, collaborative environment.
  • Strong leadership and communication skills, with the ability to mentor and guide team members.
  • Experience with version control systems (e.g., Git) and agile development methodologies.
  • Knowledge of data security and privacy best practices is a plus.

Education: B.E/B.Tech (CS/IT) preferred

Location: Noida 135 (WFO)

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Lead Data Engineer
Lead Data Engineer

Ample Success HR • Pune District

On-site
INR 2,500,000 - 4,500,000
Lead Data Engineer
Lead Data Engineer

Weekday AI • Delhi

On-site
INR 400,000 - 600,000
Lead Data Engineer (Lead Software Engineer)
Lead Data Engineer (Lead Software Engineer)

DigitalXNode • Ahmedabad District

On-site
INR 1,800,000 - 2,600,000
Lead Software Engineer - Data
Lead Software Engineer - Data

EPAM Systems Inc • Bengaluru

On-site
INR 4,000,000 - 7,000,000
Data Engineer_Spark/Scala
Data Engineer_Spark/Scala

Zorba AI • Maharashtra

On-site
INR 800,000 - 1,200,000
Lead Data Engineer
Lead Data Engineer

Eucloid Data Solutions Pvt Ltd • Chennai

On-site
INR 2,500,000 - 4,200,000
Data Engineer_Spark/Scala
Data Engineer_Spark/Scala

Zorba AI • Mumbai

On-site
INR 1,000,000 - 1,500,000
Senior Data Engineer – Data Engineering & Infrastructure – Remote India (5-10 Years Exp )
Senior Data Engineer – Data Engineering & Infrastructure – Remote India (5-10 Years Exp )

AppSierra Solutions • India

Remote
INR 3,000,000 - 6,000,000
Data Engineer
Data Engineer

Idexcel • Bengaluru

On-site
INR 900,000 - 1,400,000
Lead Data Engineer
Lead Data Engineer

Forward Eye Technologies • Dadri

On-site
INR 1,800,000 - 3,200,000