Databricks - Technical Lead-Data Engg

Birlasoft; Pune District; On-site; INR 900,000 - 1,300,000; Full time

Job summary

Birlasoft seeks a data engineer with strong Azure Databricks, PySpark, and SQL skills to design, develop, and deploy scalable data pipelines for complex data integration projects in India. You will build modern data warehouse solutions using Azure Data Lake and Databricks and collaborate with BI teams to deliver insights.

Minimum 2–3 years in Azure Databricks, ADF, and PySpark, with hands-on experience processing structured and non-structured data across multiple data sources.

Qualifications

• Experience in designing and deploying solutions using Azure Databricks, PySpark, SQL, and Data Factory.
• Develop scalable data pipelines and integrate new data sources to handle growing data volumes.
• Create Technical Specification Design and Application Interface Design.
• Develop modern Data Warehouse solutions using Azure Data Lake, Azure Databricks, and PySpark.

Responsibilities

• Collaborate with BI & analytics and business teams to translate requirements into data solutions.
• Develop batch processing and integration solutions for structured and unstructured data.
• Demonstrate in-depth Databricks and PySpark skills and optimize SQL queries.

Tools

Databricks

PySpark

SQL

Azure Data Factory

Azure Data Lake

Azure Databricks

Job description

Job Description:

Area(s) of responsibility

• Experience in Perform Design, Development & Deployment using Azure Services (Databricks, PySpark, SQL, Data Factory,)
• Develop and maintain scalable data pipelines and build new Data Source integrations to support increasing data volume and complexity.
• Experience in creating Technical Specification Design, Application Interface Design.
• Developing Modern Data Warehouse solutions using Azure Stack (Azure Data Lake, Azure Databricks) and PySpark)
• Develop batch processing and integration solutions and process Structured and Non-Structured Data
• Demonstrated in-depth skills with Databricks and PySpark, and SQL
• Collaborate and engage with BI & analytics and the business team.

• Minimum 2 to 3 year of Project experience in Azure Databricks
• Minimum 2-3 years of experience in ADF
• Minimum 2+ years of experience in PySpark

Location: Mumbai, Pune, Noida, Bangalore