Azure Databricks Developer

Birlasoft; Dadri; On-site; INR 1,500,000 - 3,000,000; Full time

Job summary

Birlasoft is hiring a Senior Data Platform Engineer in India to build and scale enterprise-grade data pipelines on Azure and Databricks. You will own production-grade solutions, implement Delta Lake patterns, and enforce governance while mentoring peers.

The role focuses on building scalable data solutions, optimizing pipelines, and driving standards across engineering teams in a fast-paced environment.

Qualifications

• Senior-level data platform engineering experience.
• Production-scale data solutions on Azure Databricks.
• Hands-on experience with data pipelines and governance.

Responsibilities

• Build scalable pipelines using Databricks and Azure services.
• Extend metadata-driven ingestion frameworks.
• Develop modular PySpark code and libraries.
• Implement Delta Lake patterns (CDC, SCD Type 2).
• Optimize SQL workloads for performance and cost.
• Orchestrate pipelines via Synapse or UC4.
• Apply data modeling principles (Kimball, 3NF, lakehouse).
• Implement CI/CD with Azure DevOps.
• Enforce governance via Unity Catalog.
• Mentor engineers and contribute to standards.

Skills

Databricks

Azure

PySpark

Delta Lake

SQL optimization

CI/CD

Data governance

Unity Catalog

Mentoring

Education

Bachelor's degree in Computer Science or related field

Tools

DLT

Autoloader

Workflows

Synapse

UC4

Job description

About the Role

We are seeking a Senior Data Platform Engineer with extensive hands‑on experience to build and scale enterprise‑grade data pipelines on Azure and Databricks. This role requires deep technical expertise, strong ownership, and the ability to independently deliver complex, production‑ready data solutions while setting engineering best practices.

What You'll Do

• Build scalable pipelines using Databricks (DLT, Autoloader, Workflows) and Azure services
• Extend metadata‑driven ingestion frameworks (declarative, reusable)
• Develop modular PySpark code and libraries (WHL packaging)
• Implement Delta Lake patterns (CDC, SCD Type 2, merges)
• Optimize SQL workloads for performance and cost
• Orchestrate pipelines via Synapse or UC4
• Apply data modeling (Kimball, 3NF, lakehouse)
• Implement CI/CD with Azure DevOps
• Enforce governance via Unity Catalog
• Mentor engineers and contribute to engineering standards

Required Technical Skills & Proficiency

Cloud Platform: Azure (ADLS Gen2, Key Vault, Azure SQL, Entra ID) – Advanced (deep hands‑on, production‑scale design & troubleshooting)