Databricks Tech Lead

Seikor Group; Hyderabad, Bengaluru; Hybrid; INR 4,000,000 - 7,000,000; Full time

Job summary

Seikor Group is seeking a Databricks Tech Lead in India (Bangalore & Hyderabad) with 9+ years of data engineering experience. You will lead migration from Hadoop to Azure, design scalable data pipelines, and collaborate with data analysts and scientists to deliver reliable datasets across portfolios.

The role requires strong hands-on expertise in Azure, Azure Databricks, SQL, Spark, and Scala/Python, along with leadership capabilities to guide cross-functional teams.

Qualifications

• 9+ years of experience in data engineering / data platforms with strong leadership.

Responsibilities

• Design and build scalable, efficient, reliable data pipelines for ETL to data lakes/warehouses.
• Provide technical leadership for migration from Hadoop to Azure.
• Develop data models to optimize storage, retrieval, and analysis.
• Integrate data from databases, APIs, streaming platforms, and third-party systems.
• Apply data transformation and cleansing to ensure data quality.
• Design and maintain data warehouses and data lakes for large data volumes.
• Implement data governance and security measures for privacy/compliance.
• Monitor and optimize data processing and query performance; resolve bottlenecks.
• Collaborate with Data Analysts and Data Scientists to provide reliable datasets.
• Provide technical guidance and collaborate with cross-functional teams.

Skills

Python

SQL

Scala

Azure Cloud

Azure Databricks

Apache Spark

Data Pipelines

ETL

Data Modeling

Data Governance

Hadoop

Kafka

Airflow

Team Leadership

Cloud Platforms

Tools

Jira

GitHub

DevOps tools

Job description

Databricks Tech Lead

Experience: 9+ years
Location: Bangalore & Hyderabad

Role Overview

We are looking for an experienced Databricks Tech Lead with 9+ years of experience in data engineering, data pipeline development, cloud data platforms, and big data technologies. The role will provide technical leadership for the migration from Hadoop to Azure and help drive how data is leveraged across multiple portfolios.

The ideal candidate should have strong hands-on experience with Azure, Azure Databricks, SQL, Spark, and Scala/Python, along with strong data engineering and technical leadership capabilities.

Key Responsibilities

• Design and build scalable, efficient, and reliable data pipelines to extract, transform, and load (ETL) data from various sources into data lakes or data warehouses.
• Provide technical leadership for the migration from Hadoop to Azure and guide how data can be effectively leveraged across multiple portfolios.
• Develop and implement data models that optimize data storage, retrieval, and analysis, considering data volume, variety, and velocity.
• Integrate data from multiple sources, including databases, APIs, streaming platforms, and third-party systems, ensuring data consistency and accuracy.
• Apply data transformation and cleansing techniques to clean, preprocess, and enrich data, ensuring data quality and readiness for analysis and reporting.
• Design and maintain data warehouses and data lakes to efficiently store and manage large volumes of structured and unstructured data.
• Implement data governance practices and security measures to protect sensitive data, ensure data privacy, and support compliance requirements such as GDPR and CCPA.
• Monitor and optimize data processing and query performance, identify bottlenecks, and implement solutions to improve data processing and retrieval efficiency.
• Collaborate with Data Analysts and Data Scientists to understand requirements and provide reliable, accurate, and well-organized datasets for analysis and modeling.
• Provide technical guidance and collaborate with cross-functional teams to resolve complex data engineering challenges.

Required Technical Skills

• 9+ years of experience in data engineering / data platforms, with strong technical leadership experience.
• Strong programming and scripting skills in Python, SQL, or Scala.
• Hands‑on experience with:
• Azure Cloud
• Azure Databricks
• SQL
• Apache Spark
• Scala and/or Python
• Experience with Jira, GitHub, and DevOps tools.
• Strong experience in data integration and ETL.
• Familiarity with technologies and frameworks such as Apache Kafka, Apache Spark, Apache Airflow, or Talend.
• Strong understanding of data modeling, dimensional modeling, and database/schema design.
• Experience with big data technologies such as Apache Hadoop, Apache Hive, Apache HBase, or Apache Cassandra.
• Understanding of cloud platforms and related services for data storage, processing, and analytics.
• Knowledge of data governance, data security, privacy, and regulatory compliance.

Key Competencies

• Strong problem-solving and analytical thinking skills.
• Ability to identify and resolve data-related issues and optimize data pipelines.
• Strong attention to detail and focus on data quality, reliability, and scalability.
• Excellent communication and collaboration skills.
• Ability to work effectively with data analysts, data scientists, engineers, and business stakeholders.
• Ability to explain complex technical concepts clearly to non-technical stakeholders.