Lead Data Engineer

Tudiptech Tudip Technologies Pvt Ltd; Pune District; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

A prominent technology firm in Pune District is seeking an experienced Lead Data Engineer to design and manage scalable data solutions. The ideal candidate will lead a data engineering team, ensuring high-quality data pipelines using Databricks. A Bachelor's or Master's degree in Computer Science or Engineering, along with a minimum of 7 years of data engineering experience, is required. Applicants must possess strong skills in Apache Spark, SQL, and experience with cloud platforms. Leadership skills are essential, and this role offers the opportunity to work in a dynamic and innovative technology environment.

Qualifications

• 7+ years of experience in data engineering or related roles.
• Mandatory hands-on experience with Databricks (Delta Lake, Databricks SQL, notebooks, jobs, workflows).
• Strong experience with Apache Spark and distributed data processing.

Responsibilities

• Lead the design, development, and maintenance of robust and scalable data pipelines using Databricks.
• Architect end-to-end data solutions including data ingestion, transformation, storage, and analytics.
• Guide and mentor a team of data engineers, ensuring best practices and code quality.

Skills

Data Engineering

Databricks

Apache Spark

SQL

Python

Cloud Platforms (AWS, Azure, GCP)

Data Warehousing

ETL/ELT Tools

DevOps Practices

Leadership

Education

Bachelor’s or Master’s degree in Computer Science or Engineering

Tools

Databricks SQL

Snowflake

Redshift

BigQuery

Apache Airflow

Job description

About Company:

Tudip Technologies Pvt. Ltd is a CMMI Level 5 extreme technology company. Careers at Tudip Technologies are not just jobs, but a promise of a bright and dynamic future. Tudip provides ample opportunities to grow within the company technically as well as a technocrat by promoting entrepreneurship. Tudip Technologies’ careers will enable you to help clients enhance and improve while you build your career. We are a place which defines Integrity, Innovation, and Serenity. Tudip provides you a better platform that transforms an individual into an experienced and immensely skilled professional through an ethical and vibrant business environment. We are here for effective client servicing, taking care of our employees’ needs, and creating a success story to remember.

Position Summary:

We are looking for an experienced Lead Data Engineer to design, build, and manage scalable data pipelines and platforms. The ideal candidate will lead the data engineering team, collaborate with cross-functional stakeholders, and ensure high-quality, reliable, and secure data solutions that support business and analytics needs. Hands-on experience with Databricks is mandatory for this role.

Please read the job criteria below and drop us an email at [e-mail] OR create an account at our Recruitment Portal to get started.

Key Responsibilities

• Lead the design, development, and maintenance of robust and scalable data pipelines using Databricks.
• Architect end-to-end data solutions including data ingestion, transformation, storage, and analytics.
• Build and optimize data workflows using Databricks, Spark, and cloud data platforms.
• Guide and mentor a team of data engineers, ensuring best practices and code quality.
• Collaborate with product managers, data scientists, analysts, and business teams to understand data requirements.
• Ensure data quality, governance, security, and compliance standards are met.
• Optimize performance and cost of data pipelines and platforms.
• Review technical designs, code, and deployments.
• Troubleshoot and resolve complex data-related issues.
• Participate in strategic planning and roadmap discussions for data platforms.

Required Skills & Qualifications

• Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
• 7+ years of experience in data engineering or related roles.
• Mandatory hands-on experience with Databricks (Delta Lake, Databricks SQL, notebooks, jobs, workflows).
• Strong experience with Apache Spark and distributed data processing.
• Proficiency in SQL and programming languages such as Python, Scala, or Java.
• Experience with cloud platforms (AWS, Azure, or GCP).
• Expertise in data warehousing solutions (Snowflake, Redshift, BigQuery, etc.).
• Experience with ETL/ELT tools and orchestration frameworks (Airflow, dbt, etc.).
• Strong understanding of CI/CD, version control, and DevOps practices.
• Excellent leadership, communication, and problem-solving skills.

Preferred Skills

• Experience leading or mentoring data engineering teams.
• Knowledge of real-time data processing and streaming (Kafka, Spark Streaming, etc.).
• Exposure to BI and analytics tools.
• Experience working in Agile/Scrum environments.