Lead Data Engineer

Tudiptech Tudip Technologies Pvt Ltd; Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

A leading technology company in Pune District is seeking an experienced Lead Data Engineer to design and manage scalable data pipelines. Responsibilities include leading a team, collaborating with stakeholders, and ensuring data quality. The ideal candidate has over 7 years of experience in data engineering, strong skills in Databricks, and expertise in cloud platforms. This position offers opportunities for professional growth in a dynamic environment.

Qualifications

• 7+ years of experience in data engineering or related roles.
• Mandatory hands-on experience with Databricks, including Delta Lake and workflows.
• Strong understanding of CI/CD, version control, and DevOps practices.

Responsibilities

• Lead the design, development, and maintenance of data pipelines using Databricks.
• Architect end-to-end data solutions for ingestion, transformation, and storage.
• Guide and mentor a team of data engineers closely.

Skills

Databricks

Apache Spark

SQL

Python

Cloud platforms (AWS, Azure, or GCP)

ETL/ELT tools

CI/CD practices

Education

Bachelor’s or Master’s degree in Computer Science or Engineering

Tools

Snowflake

Redshift

BigQuery

Job description

About Company:

Tudip Technologies Pvt. Ltd is a CMMI Level 5 extreme technology company. Careers at Tudip Technologies are not just jobs, but a promise of a bright and dynamic future. Tudip provides ample opportunities to grow within the company technically as well as a technocrat by promoting entrepreneurship. Tudip Technologies’ careers will enable you to help clients enhance and improve while you build your career. We are a place which defines Integrity, Innovation, and Serenity. Tudip provides you a better platform that transforms an individual into an experienced and immensely skilled professional through an ethical and vibrant business environment. We are here for effective client servicing, taking care of our employees’ needs, and creating a success story to remember.

Position Summary:

We are looking for an experienced Lead Data Engineer to design, build, and manage scalable data pipelines and platforms. The ideal candidate will lead the data engineering team, collaborate with cross-functional stakeholders, and ensure high-quality, reliable, and secure data solutions that support business and analytics needs. Hands-on experience with Databricks is mandatory for this role.

Please read the job criteria below and drop us an email at [e-mail] OR create an account at our Recruitment Portal to get started.

Key Responsibilities

• Lead the design, development, and maintenance of robust and scalable data pipelines using Databricks.
• Architect end-to-end data solutions including data ingestion, transformation, storage, and analytics.
• Build and optimize data workflows using Databricks, Spark, and cloud data platforms.
• Guide and mentor a team of data engineers, ensuring best practices and code quality.
• Collaborate with product managers, data scientists, analysts, and business teams to understand data requirements.
• Ensure data quality, governance, security, and compliance standards are met.
• Optimize performance and cost of data pipelines and platforms.
• Review technical designs, code, and deployments.
• Troubleshoot and resolve complex data-related issues.
• Participate in strategic planning and roadmap discussions for data platforms.

Required Skills & Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• 7+ years of experience in data engineering or related roles.
• Mandatory hands-on experience with Databricks (Delta Lake, Databricks SQL, notebooks, jobs, workflows).
• Strong experience with Apache Spark and distributed data processing.
• Proficiency in SQL and programming languages such as Python, Scala, or Java.
• Experience with cloud platforms (AWS, Azure, or GCP).
• Expertise in data warehousing solutions (Snowflake, Redshift, BigQuery, etc.).
• Experience with ETL/ELT tools and orchestration frameworks (Airflow, dbt, etc.).
• Strong understanding of CI/CD, version control, and DevOps practices.
• Excellent leadership, communication, and problem-solving skills.

Preferred Skills

• Experience leading or mentoring data engineering teams.
• Knowledge of real-time data processing and streaming (Kafka, Spark Streaming, etc.).
• Exposure to BI and analytics tools.
• Experience working in Agile/Scrum environments.