Senior Azure Data Engineer

S2Integrators; India; On-site; INR 6,369,426 - 8,189,262; Full time

Job summary

A leading IT services company is seeking a Senior Azure Data Engineer in Bangalore. This role requires 4–7 years of experience in building scalable data pipelines with a focus on Azure Databricks and data orchestration using Azure Data Factory. The ideal candidate will have strong SQL skills and knowledge of ADLS Gen2. The position is full-time and emphasizes collaboration with analytics teams to ensure data quality and performance optimization.

Qualifications

• 4–7 years of experience in Data Engineering.
• Strong hands‑on experience with Azure Databricks.
• Proficiency in PySpark and Spark SQL.
• Experience with Azure Data Factory (ADF) for orchestration.
• Hands‑on experience with ADLS Gen2.

Responsibilities

• Design, develop, and maintain end‑to‑end batch and incremental data pipelines on Azure.
• Build ETL/ELT workflows using Azure Databricks.
• Orchestrate pipelines using Azure Data Factory (ADF).
• Implement and manage Delta Lake.
• Collaborate with analytics and BI teams.

Skills

Azure Databricks

PySpark

Spark SQL

Azure Data Factory (ADF)

ADLS Gen2

SQL

Tools

Git

Job description

Role Overview

We are seeking a Senior Azure Data Engineer (4–7 years) with hands‑on experience in building, optimizing, and supporting scalable data pipelines on the Microsoft Azure platform. The role emphasizes Azure Databricks, ADF, and ADLS, with ownership of end‑to‑end data engineering solutions in production environments.

Key Responsibilities

• Design, develop, and maintain end‑to‑end batch and incremental data pipelines on Azure
• Build ETL/ELT workflows using Azure Databricks (PySpark, Spark SQL)
• Orchestrate pipelines using Azure Data Factory (ADF)
• Implement and manage Delta Lake (ACID, schema evolution, time travel)
• Optimize Spark jobs for performance, scalability, and cost
• Integrate data from RDBMS, files, APIs, and cloud storage
• Ensure data quality, monitoring, and reliability in production
• Collaborate with analytics, BI, and downstream consumers
• Participate in code reviews and mentor junior engineers

Required Skills

Core Technical Skills

• 4–7 years of experience in Data Engineering
• Strong hands‑on experience with Azure Databricks
• Proficiency in PySpark and Spark SQL
• Experience with Azure Data Factory (ADF) for orchestration
• Hands‑on experience with ADLS Gen2
• Strong SQL skills and understanding of data modeling

Azure & Cloud

• Azure Storage, Azure Key Vault
• Basic understanding of Azure security (AAD, RBAC)
• Exposure to Azure Synapse (preferred)

Tools & Practices

• Git‑based version control
• Workflow scheduling and monitoring
• Performance tuning and cost optimization

Good to Have

• Experience with streaming data (Kafka / Event Hubs / Spark Streaming)
• ML exposure using MLflow / Databricks ML
• Unity Catalog & data governance
• Python scripting beyond Spark

Seniority level

Mid‑Senior level

Employment type

Full‑time

Job function

Information Technology

Industries

IT Services and IT Consulting

Location

Bangalore Urban, Karnataka, India