Senior Data Engineer (Databricks Lead)

Prodapt Solutions Private Limited

Bengaluru

On-site

INR 4,000,000 - 7,000,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Prodapt Solutions Private Limited seeks a Senior Data Engineer (Databricks Lead) with 12-16 years of experience to drive design, development, and optimization of enterprise-scale data platforms.

The role emphasizes Databricks, Spark, Snowflake, Kafka, Python, SQL, and AWS to enable analytics, AI, and real-time processing.

Qualifications

  • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
  • 12-16 years of experience in Data Engineering and Data Platform Architecture.
  • Proven experience leading enterprise-scale data modernization and cloud migration initiatives.

Responsibilities

  • Lead architecture, design, and implementation of scalable data platforms using Databricks, Snowflake, and AWS.
  • Design and optimize large-scale batch and real-time data pipelines using Spark and Kafka.
  • Develop and maintain robust ETL/ELT frameworks for data reliability and performance.

Skills

Databricks
Apache Spark
Snowflake
Kafka
Python
SQL
AWS

Education

Bachelor's or Master's in CS/IT/Engineering

Tools

REST APIs
GraphQL APIs
Databricks APIs

Job description

Overview

We are looking for a Senior Data Engineer (Databricks Lead)with 12-16 years of experience to lead the design, development, and optimization of enterprise-scale data platforms. The ideal candidate will have deep expertise in Databricks, Apache Spark, Snowflake, Kafka, Python, SQL, and AWS, along with experience building scalable data architectures that support analytics, real-time processing, AI/ML, and Generative AI initiatives..

Responsibilities
  • Lead the architecture, design, and implementation of scalable, secure, and high-performance data platforms using Databricks, Snowflake, and AWS.
  • Design and optimize large-scale batch and real-time data pipelines using Spark and Kafka.
  • Develop and maintain robust ETL/ELT frameworks to ensure data reliability, scalability, and performance.
  • Implement real-time data ingestion and processing solutions using Kafka and Spark Streaming.
  • Build automated data integration frameworks leveraging REST APIs, GraphQL APIs, and Databricks APIs.
  • Design cloud-native data solutions aligned with enterprise governance, security, and compliance standards.
  • Develop data models using Star Schema, Snowflake Schema, Kimball, and Inmon methodologies.
  • Provide technical leadership, architecture guidance, and mentorship to engineering teams.
  • Collaborate with Analytics, Data Science, and AI teams to enable Machine Learning, Generative AI, and LLM-based solutions.
  • Troubleshoot and resolve platform performance bottlenecks and scalability challenges.
  • Drive engineering best practices, code quality standards, and data governance initiatives.
Requirements
Technical Skills
  • Strong expertise in Databricks and Apache Spark.
  • Hands-on experience with Snowflake and modern data warehouse architectures.
  • Proficiency in Kafka and real-time stream processing.
  • Strong programming skills in Python and advanced SQL.
  • Experience with AWS Cloud Services.
  • Expertise in designing and implementing ETL/ELT architectures.
  • Strong understanding of data modeling concepts and methodologies.
  • Experience integrating systems using REST and GraphQL APIs.
Qualifications
  • Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field.
  • 12-16 years of experience in Data Engineering and Data Platform Architecture.
  • Proven experience leading enterprise-scale data modernization and cloud migration initiatives.
Preferred Skills
  • Experience with AI/ML data platforms, Generative AI, and LLM enablement.
  • Exposure to Data Governance and Data Quality frameworks.
  • Strong stakeholder management, communication, and technical leadership skills.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Data Engineer (Databricks Lead)
Senior Data Engineer (Databricks Lead)

Prodapt Solutions • Bengaluru

On-site
INR 4,000,000 - 8,000,000
Senior Data Engineer (Databricks Lead)
Senior Data Engineer (Databricks Lead)

Prodapt • Bengaluru

On-site
INR 4,000,000 - 7,000,000
Senior Databricks Engineer
Senior Databricks Engineer

Prodapt • Bengaluru

On-site
INR 3,500,000 - 5,500,000
Big Data Architect
Big Data Architect

Minfy Technologies • Hyderabad

On-site
INR 4,000,000 - 7,000,000
Senior Data Engineer
Senior Data Engineer

Amgen Technology • Hyderabad

Hybrid
INR 2,500,000 - 4,000,000
Senior Data Engineer Databricks
Senior Data Engineer Databricks

RITWIK IT SERVICES PRIVATE LIMITED • Bengaluru

Hybrid
INR 2,500,000 - 4,000,000
Sr. Databricks Engineer
Sr. Databricks Engineer

Fa Ewjt Saasfaprod1 • Pune District

Hybrid
INR 3,500,000 - 6,000,000
Senior Data Architect
Senior Data Architect

JK Tech • Bengaluru

On-site
INR 2,000,000 - 3,000,000
Databricks Engineer
Databricks Engineer

Impronics Technologies • Gurugram District

On-site
INR 4,000,000 - 7,000,000
Senior Manager - Data Engineering (Databricks)
Senior Manager - Data Engineering (Databricks)

Bread Financial • Bengaluru

Hybrid
INR 4,500,000 - 6,500,000