IN_Manager_Azure Databricks_D&A_Advisory_Bangalore

PwC India; Bengaluru; On-site; INR 4,200,000 - 7,000,000; Full time

Job summary

PwC India is seeking a Senior Databricks Engineer (Manager designation) in Bengaluru to design and implement Databricks Lakehouse architectures across Bronze/Silver/Gold medallions. You will build reusable ingestion and transformation frameworks and lead large-scale ETL pipelines with strong SQL and data modeling skills.

You will drive CI/CD and DevOps automation, RBAC with Unity Catalog, and performance tuning for cloud data platforms on AWS/Azure/GCP.

Qualifications

• B.E/B.Tech in CS/IT required.
• 5–6+ years in Data Engineering; 3+ years in Databricks.
• Deep expertise in SparkSQL, PySpark, Python, Delta Lake.
• Strong cloud experience on AWS/Azure/GCP.
• CI/CD for data platforms; ETL design experience.
• Strong SQL and data modeling skills.

Responsibilities

• Design Databricks Lakehouse medallion architecture.
• Architect reusable ingestion & transformation frameworks.
• Prepare physical data models.
• Design and develop ingestion pipelines from RDBMS, API, files, etc.
• Transform structured, unstructured, and semi-structured data.
• Define CI/CD and DevOps automation strategy.
• Implement RBAC via Unity Catalog.
• Define clustering, scaling and performance strategies.

Skills

Databricks

SparkSQL

PySpark

Python

SQL

ETL design

Data modeling

CI/CD

Performance tuning

Education

B.E./B.Tech.

MBA

M.E./M.Tech

Tools

Azure

AWS

GCP

Terraform

CloudFormation

Job description

At PwC, our people in data and analytics focus on leveraging data to drive insights and make informed business decisions. They utilise advanced analytics techniques to help clients optimise their operations and achieve their strategic goals.

In business intelligence at PwC, you will focus on leveraging data and analytics to provide strategic insights and drive informed decision-making for clients. You will develop and implement innovative solutions to optimize business performance and enhance competitive advantage.

Job Position Title

IN_Manager_Azure Databricks_D&A_Advisory_Bangalore

Responsibilities

Role Name: Senior Databricks Engineer

Designation : Manager

Required Skills

• B.Tech/B.E degree in Computer Science or Information Technology.
• 5-6+ years in Data Engineering; 3+ years in Databricks
• Deep expertise in SparkSQL, PySpark, Python, Delta Lake
• Strong cloud experience (AWS / Azure / GCP)
• CI/CD implementation for data platforms
• Strong performance tuning and workload optimization
• Experience designing large-scale ETL pipeline
• Strong understanding on SQL and data modeling skills

Good to Have

• Experience with GenAI/Agentic AI for code conversion, automation, or metadata generation
• Exposure to Infrastructure as Code (Terraform/CloudFormation)
• Experience integrating on-prem DBs (Teradata/Oracle) with cloud

Key Responsibilities

• Design Databricks Lakehouse medallion architecture (Bronze/Silver/Gold)
• Architect reusable ingestion & transformation frameworks using Databricks and Enterprise level ETL / orchestration tool.
• Preparing Physical data Model
• Design and develop ingestion pipelines for various type of sources – RDBMS, API, Files, etc.
• Experience in writing code to transform variety data formats (structured, unstructured, and semi-structured) to uniform formats.
• Define CI/CD and DevOps automation strategy
• Implement RBAC for various user persona by Unity catalog
• Define clustering, scaling, and performance strategies

Mandatory skill sets

Azure Databricks

Preferred skill sets

Azure Databricks

Years of experience required

8—12 Years

Education qualification

B.E/B.Tech/MBA/M.E/M.Tech