Databricks + PySpark - Technical Lead-Data Engg

Birlasoft; Pune District; On-site; INR 3,500,000 - 6,000,000; Full time

Job summary

Birlasoft is seeking an experienced Databricks + PySpark - Technical Lead for Data Engineering in India. You will design, develop and deploy scalable data pipelines using Azure services including Databricks, PySpark, SQL and Azure Data Factory.

You will build modern data warehouse solutions on Azure Stack, develop batch processing, integrate data sources, and collaborate with BI/analytics teams to deliver insights. Location options include Mumbai, Pune, Noida, and Bengaluru with on-site work.

Qualifications

• 2-3 years of project experience in Azure Databricks.
• 2-3 years of experience in Azure Data Factory (ADF).
• 2+ years of experience in PySpark.

Responsibilities

• Design, develop & deploy scalable data pipelines using Azure services (Databricks, PySpark, SQL, Data Factory).
• Develop and maintain data pipelines and build new data source integrations to support increasing data volume and complexity.
• Develop modern data warehouse solutions using Azure Data Lake, Azure Databricks and PySpark.
• Develop batch processing and integration solutions and process structured and unstructured data.
• Collaborate with BI/analytics teams and business stakeholders.

Skills

Databricks

PySpark

SQL

Azure Data Factory

ETL pipelines

Data integration

Data modeling

Job description

Select how often (in days) to receive an alert:

Location: INDIA - PUNE - BIRLASOFT OFFICE - HINJAWADI

Title: Databricks + PySpark - Technical Lead-Data Engg

Description:

Area(s) of responsibility

• Experience in Perform Design, Development & Deployment using Azure Services (Databricks, PySpark, SQL, Data Factory,)
• Develop and maintain scalable data pipelines and build new Data Source integrations to support increasing data volume and complexity.
• Experience in creating Technical Specification Design, Application Interface Design.
• Developing Modern Data Warehouse solutions using Azure Stack (Azure Data Lake, Azure Databricks) and PySpark
• Develop batch processing and integration solutions and process Structured and Non-Structured Data
• Demonstrated in-depth skills with Databricks and PySpark, and SQL
• Collaborate and engage with BI & analytics and the business team.

• Minimum 2 to 3 year of Project experience in Azure Databricks
• Minimum 2-3 years of experience in ADF
• Minimum 2+ years of experience in PySpark

Location: Mumbai, Pune, Noida, Bangalore