Lead Data Databricks

Perficient India; Hyderabad; Hybrid; INR 1,400,000 - 2,100,000; Full time

Benefits offered by this job

Hybrid work model

Job summary

Perficient India is seeking a Lead Azure Databricks Engineer to design, build, and optimize scalable data pipelines on Azure. The role demands 7+ years of data engineering experience with hands-on Azure Databricks, PySpark, SQL and Python.

The position is full-time with a hybrid work model (3 days in office) and locations including Hyderabad. Strong problem-solving and communication skills are essential.

Qualifications

• 7+ years of experience in data engineering.
• Strong hands-on in Azure Databricks, PySpark, SQL and Python.
• Understanding of ETL/ELT concepts and data warehousing.

Responsibilities

• Design and maintain ETL/ELT pipelines using Azure Databricks.
• Write efficient PySpark and Python code for large-scale data processing.
• Develop and optimize SQL queries for data transformation and reporting.
• Work with Delta Lake, ADLS, and Azure Data Factory.
• Monitor and troubleshoot pipeline performance.
• Collaborate with analysts and business teams to deliver reliable datasets.

Skills

Azure Databricks

PySpark

SQL

Python

Data warehousing

ETL/ELT design

Azure ecosystem

Job description

We are looking for a Lead Azure Databricks Engineer

Locations: #Hyderabad |#Bangalore | #Chennai | #Pune | #Nagpur

Experience: 9+ Years

Employment Type: Full-time | Hybrid (3 days work from office)

Interview Mode: Virtual followed by mandatory in-person

Notice Period: Immediate- 15 Days

Job Title: Azure Databricks Engineer (Lead or Architect)

Job Description:

We are looking for a skilled Data Engineer with strong expertise in Azure Databricks, PySpark, SQL, and Python to join our team. The ideal candidate will design, build, and optimize scalable data pipelines and support analytics initiatives on the Azure platform.

Responsibilities:

• Design and maintain ETL/ELT pipelines using Azure Databricks
• Write efficient PySpark and Python code for large-scale data processing
• Develop and optimize complex SQL queries for data transformation and reporting
• Work with Delta Lake, Azure Data Lake Storage (ADLS), and Azure Data Factory
• Monitor and troubleshoot pipeline performance
• Collaborate with data analysts and business teams to deliver reliable datasets

Requirements:

• 7+ years of experience in data engineering
• Strong hands-on skills in Azure Databricks, PySpark, SQL, and Python
• Understanding of data pipeline design, ETL/ELT concepts, and data warehousing
• Familiarity with Azure ecosystem tools is a plus
• Good problem-solving and communication skills

Preferred Certifications

Databricks Certified Data Engineer Associate or Professional