Lead Data Engineer

Cloud Counselage Pvt Ltd; Mumbai; Hybrid; INR 1,500,000 - 2,500,000; Full time

Job summary

A technology services company is seeking an experienced Lead Data Engineer to lead and mentor a skilled team of data engineers. In this role, you will drive the design and implementation of scalable data solutions and optimize data pipelines, enhancing the company's data infrastructure. Candidates should have over 7 years of experience in data engineering, strong leadership skills, and proficiency in data programming languages. The position offers a flexible remote, hybrid, or in-office work model.

Qualifications

• 7+ years of proven experience in a data engineering role.
• Demonstrated expertise in designing and implementing scalable data solutions.
• Excellent leadership and communication skills.

Responsibilities

• Provide technical leadership and mentorship to a team of data engineers.
• Lead the design and implementation of scalable data architectures.
• Oversee the development and optimization of ETL pipelines.
• Define and implement data models and schemas.
• Establish data governance policies to ensure data quality.

Skills

Python

SQL

Java

Data manipulation

Data analysis tools

Education

Bachelor's or master's degree in Computer Science, Engineering, Information Systems

Tools

Apache Spark

SQL databases

NoSQL databases

ETL frameworks

AWS

Azure

GCP

Docker

Kubernetes

Job description

We are seeking an experienced and dynamic Lead Data Engineer to join our team. This role offers a unique opportunity to lead and mentor a talented team of data engineers, drive the design and implementation of scalable data solutions, and play a key role in shaping our data infrastructure and strategy.

Responsibilities

• Team Leadership: Provide technical leadership and mentorship to a team of data engineers, fostering a collaborative and innovative environment focused on delivering high‑quality solutions.
• Architectural Design: Lead the design and implementation of scalable, robust, and maintainable data architectures, ensuring alignment with business goals and future scalability requirements.
• Data Pipeline Development: Oversee the development and optimization of ETL pipelines to extract, transform, and load data from diverse sources into data warehouses and other storage systems.
• Data Modeling: Define and implement data models and schemas to support business requirements, optimizing for performance, flexibility, and scalability.
• Data Governance and Quality: Establish and enforce data governance policies and best practices to ensure data quality, integrity, and security across all data assets.
• Technological Innovation: Stay abreast of emerging technologies and trends in data engineering and analytics, evaluating their potential impact and leading initiatives to adopt relevant technologies.
• Cross‑Functional Collaboration: Collaborate closely with data scientists, analysts, and business stakeholders to understand data requirements, deliver actionable insights, and drive data‑driven decision‑making.
• Performance Optimization: Monitor and optimize data pipelines, databases, and data processing systems for performance, scalability, and efficiency, identifying and addressing bottlenecks and optimization opportunities.
• Documentation and Best Practices: Develop and maintain documentation, standards, and best practices for data engineering processes, ensuring knowledge sharing and maintainability.
• Project Management: Lead data engineering projects from conception to completion, including project planning, resource allocation, and stakeholder communication.

Qualifications

• Bachelor's or master's degree in Computer Science, Engineering, Information Systems, or a related field.
• 7+ years of proven experience in a data engineering role, with demonstrated expertise in designing and implementing scalable data solutions.
• Strong proficiency in programming languages such as Python, SQL, or Java, as well as experience with data manipulation and analysis tools (e.g., Pandas, Apache Spark).
• Extensive experience with database technologies (e.g., SQL databases, NoSQL databases, data warehouses) and ETL frameworks.
• Excellent leadership and communication skills, with the ability to inspire and mentor a team, collaborate effectively with cross‑functional stakeholders, and influence technical decision‑making.
• Strong problem‑solving and analytical abilities, with a focus on delivering practical, scalable solutions to complex data engineering challenges.
• Experience with cloud platforms (e.g., AWS, Azure, GCP) and containerization technologies (e.g., Docker, Kubernetes) is a plus.

Job Location

Remote/Hybrid/In‑Office