Databricks + PySpark - Technical Lead-Data Engg

Birlasoft; India; On-site; INR 1,800,000 - 3,400,000; Full time

Job summary

Birlasoft is seeking a Technical Lead-Data Engg with expertise in Databricks and PySpark to design, develop, and deploy scalable data pipelines using Azure services. The role involves building modern data warehouse solutions and collaborating with BI teams to support business insights.

Candidates should have 2–3 years in Azure Databricks, 2–3 years in ADF, and 2+ years in PySpark, with locations in Mumbai, Pune, Noida, or Bangalore.

Qualifications

• Minimum 2–3 years of project experience in Azure Databricks.
• Minimum 2–3 years of experience in Azure Data Factory (ADF).
• Minimum 2+ years of experience in PySpark.

Responsibilities

• Develop and maintain scalable data pipelines using Azure Databricks and PySpark.
• Design technical specifications and interface designs.
• Build modern data warehouse solutions with Azure Data Lake and Databricks.
• Develop batch processing and integrate structured and unstructured data.
• Collaborate with BI and analytics teams and business stakeholders.

Skills

Azure Databricks

PySpark

SQL

Azure Data Factory

Tools

Azure Data Factory

Azure Databricks

Job description

Select how often (in days) to receive an alert:

Location: INDIA - PUNE - BIRLASOFT OFFICE - HINJAWADI

Title: Databricks + PySpark - Technical Lead-Data Engg

Description:

Area(s) of responsibility

• Experience in Perform Design, Development & Deployment using Azure Services (Databricks, PySpark, SQL, Data Factory,)
• Develop and maintain scalable data pipelines and build new Data Source integrations to support increasing data volume and complexity.
• Experience in creating Technical Specification Design, Application Interface Design.
• Developing Modern Data Warehouse solutions using Azure Stack (Azure Data Lake, Azure Databricks) and PySpark
• Develop batch processing and integration solutions and process Structured and Non-Structured Data
• Demonstrated in-depth skills with Databricks and PySpark, and SQL
• Collaborate and engage with BI & analytics and the business team.

• Minimum 2 to 3 year of Project experience in Azure Databricks
• Minimum 2-3 years of experience in ADF
• Minimum 2+ years of experience in PySpark

Location: Mumbai, Pune, Noida, Bangalore