Azure Databricks Lead

Birlasoft; Pune District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Birlasoft is seeking a Data Engineer with Databricks, PySpark and data modelling to design, develop and deploy scalable data solutions across Azure stack. You will build robust data pipelines and integrate diverse data sources to support complex analytics.

You will create technical specifications and interface designs, develop data warehouse solutions with Azure Data Lake and Databricks, and collaborate with BI teams to deliver business insights.

Qualifications

• Experience with design, development & deployment using Azure Services (Databricks, PySpark, SQL, Data Factory).
• Develop and maintain scalable data pipelines and integrate new data sources.
• Create technical specification and interface designs.
• Develop modern data warehouse solutions using Azure Data Lake, Databricks and PySpark.
• Handle batch processing and integration of structured and unstructured data.
• Demonstrated skills with Databricks, PySpark and SQL.
• Collaborate with BI analytics and business teams.

Responsibilities

• Design and implement data pipelines on Azure Databricks using PySpark.
• Collaborate with BI teams to translate requirements into data solutions.
• Maintain and optimize data models and data warehouses.

Skills

Databricks

PySpark

Data modelling

Job description

Area(s) of responsibility

Skills: Data Engineer with Databricks + PySpark & Data modelling

Location: Noida, Mumbai, Pune, Bangalore, Hyderabad, Chennai

Job Description

Years of experience 6 to 10 Years

• Experience in Perform Design, Development & Deployment using Azure Services (Databricks, PySpark, SQL, Data Factory,)
• Develop and maintain scalable data pipelines and build new Data Source integrations to support increasing data volume and complexity.
• Experience in creating Technical Specification Design, Application Interface Design.
• Developing Modern Data Warehouse solutions using Azure Stack (Azure Data Lake, Azure Databricks) and PySpark
• Develop batch processing and integration solutions and process Structured and Non-Structured Data
• Demonstrated in-depth skills with Databricks and PySpark, and SQL
• Collaborate and engage with BI & analytics and the business team.