Azure Databricks Engineer

Birlasoft; Pune District; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Birlasoft in Pune seeks a senior Azure Databricks Engineer with 6-12 years of experience to design and implement secure, scalable cloud data solutions using Azure Databricks, PySpark and SQL.

You will lead integration of data sources with Data Lake and Data Warehouse, optimize performance, and collaborate with clients to translate business requirements into technical architecture on AWS and Azure platforms.

Qualifications

• 6-12 years of experience in data engineering and cloud
• Expertise in Azure Databricks, PySpark and Advanced SQL
• Experience with large-scale data integration and data warehousing

Responsibilities

• Design and implement secure, scalable cloud-based data solutions on Azure
• Optimize performance and conduct tuning for large data workloads
• Integrate diverse data sources with Data Warehouse and Data Lake architectures
• Collaborate with clients to translate business requirements into technical solutions on AWS and Azure

Skills

Azure Databricks

PySpark

SQL

Python

Tools

GitHub

Jira

Teams

Confluence

Job description

Area(s) of responsibility

Skills: Azure Databricks Engineer, Python, Pyspark, SQL

Experience: 6-12 years

• Azure Lead with experience in Azure ADF, ADLS Gen2, Databricks, PySpark and Advanced SQL
• Responsible for designing and implementing secure, scalable, and highly available cloud-based solutions and estimation on Azure Cloud
• 4 Years of experience in Azure Databricks and PySpark
• Experience in Performance Tuning
• Experience with integration of different data sources with Data Warehouse and Data Lake is required
• Experience in creating Data warehouse, data lakes
• Understanding of data modelling and data architecture concepts
• To be able to clearly articulate pros and cons of various technologies and platforms
• Experience in supporting tools GitHub, Jira, Teams, Confluence need to be used
• Collaborate with clients to understand their business requirements and translate them into technical solutions that leverage AWS and Azure cloud platforms.

Mandatory Skillset: Azure Databricks, PySpark and Advanced SQL