IN_Manager_Databricks_Data And Analytics_GCC_Advisory_Bangalore(India)

Price Waterhouse Cooper LLP; Bengaluru; On-site; INR 4,000,000 - 7,000,000; Full time

Job summary

Price Waterhouse Cooper LLP in Bengaluru seeks an experienced Databricks SME/Architect to join an enterprise client engagement. The candidate will oversee Databricks platform administration, security, migrations, and integration of new NPI tools across Azure and AWS, with SAP-to-GCP/Databricks migration experience.

The role requires strong Python, PySpark, SQL and Spark skills, plus hands-on work with MLflow, Vector Search and Delta Lake.

Qualifications

• Oversee Databricks platform administration, security, maintenance, migrations, and integration of new NPI tools across Azure and AWS.
• Design, build, and optimize ETL/ELT pipelines using Apache Spark, PySpark, SQL, and Delta Lake.
• Implement Medallion Architecture using Bronze, Silver, and Gold data layers.
• Prepare datasets for generative AI and ML using MLflow and Vector Search.
• Tune Spark workloads, optimize serverless compute, and use Photonengine to improve cost and processing efficiency.
• Implement data security, access controls, governance, and lineage tracking using Databricks Unity Catalog and GCP IAM.
• Lead or support migration of SAP source-system data to GCP and Databricks.
• Engage with business stakeholders and IT delivery teams to understand functional requirements and design the end-to-end architecture.
• Provide technical input for project planning, estimation, feasibility assessment, and solution design.
• Develop optimal, scalable solutions and provide customer-facing technical leadership throughout delivery.

Responsibilities

• Oversee Databricks platform administration, security, migrations, and integration of new NPI tools across Azure and AWS.
• Design, build, and optimize ETL/ELT pipelines using Apache Spark, PySpark, SQL, and Delta Lake.
• Implement Medallion Architecture Bronze, Silver and Gold data layers.
• Prepare and manage datasets for generative AI and ML use cases using MLflow and Vector Search.
• Tune Spark workloads, optimize serverless compute, and use the Photonengine to improve cost and processing efficiency.
• Implement data security, access controls, governance, and lineage tracking using Unity Catalog and GCP IAM.
• Lead or support migration of SAP source-system data to GCP and Databricks.
• Engage with business stakeholders and IT delivery teams to understand functional requirements and design the end-to-end architecture.
• Provide technical input for project planning, estimation, feasibility assessment, and solution design.
• Develop scalable, customer-facing solutions and leadership throughout delivery.

Skills

Databricks

Azure

AWS

GCP

Apache Spark

PySpark

Python

SQL

Delta Lake

Unity Catalog

MLflow

Vector Search

Photonengine

Data Security

Governance

SAP Migration

Education

B.E./B.Tech

M.Tech

MBA

MCA

Tools

Databricks

Azure

AWS

GCP

MLflow

Vector Search

Photonengine

Unity Catalog

SAP migration tools

Job description

IN_Manager_Databricks_Data And Analytics_GCC_Advisory_Bangalore(India) Bengaluru Millenia, Bangalore, Karnataka, India 9 - 11 Years Regular

Job Description

Job Description &Summary:

We are looking for an experiencedDatabricks Subject Matter Expert (SME)/Architect to join one of ourenterprise client engagements. The candidate should have strong expertise inData Bricks Admin, Azure, AWS, GCP, and Databricks along with experience workingwith SAP source systems migrating to GCP and Databricks.

Responsibilities:

Core Focus: An experienced CloudPlatform Databricks SME, responsible for overseeing the Platformadministration, Security, new NPI tools integration, migrations, platformmaintenance and other platform administration activities on Azure/AWS
- Advanced proficiency in Python, PySpark, SQL and Apache Spark.
- Build and optimize high-performance ETL/ELT pipelines using Apache Spark,PySpark, SQL, and Delta Lake (following Medallion Architecture: Bronze,Silver, Gold layers)
- Prepare datasets for generative AI and machine learning using MLflow andVector Search.
- Hands-on experience with cloud services (primarily Azure or AWS) and CI/CDdeployment tools
- Tune Spark workloads, optimize serverless compute, and utilize Photonengine for cost and processing efficiency.
- Implement data security, access controls, and lineage tracking usingDatabricks Unity Catalog and GCP IAM
- Previous experience of migrating from SAP source systems to GCP/Databricks
- Interacting with the Business stakeholders and other IT delivery teams tounderstand the functional requirements and design the whole architecture.
- Providing technical input to projects delivering changes to support initialproject planning and estimation process
- Checking feasibility and designing and developing an optimal solution.
- Good communication skills and to be customer facing

Mandatory skill sets:

• Oversee Databricks platformadministration, security, maintenance, migrations, and integration of new NPItools across Azure and AWS.
• Design, build, and optimize high-performance ETL/ELT pipelines using ApacheSpark, PySpark, SQL, and Delta Lake.
• Implement Medallion Architecture using Bronze, Silver, and Gold datalayers.
• Prepare and manage datasets for generative AI and machine learning usecases using MLflow and Vector Search.
• Tune Spark workloads, optimize serverless compute, and use the Photonengine to improve cost and processing efficiency.
• Implement data security, access controls, governance, and lineage trackingusing Databricks Unity Catalog and GCP IAM.
• Lead or support migration of SAP source-system data to GCP and Databricks.
• Engage with business stakeholders and IT delivery teams to understandfunctional requirements and design the end-to-end architecture.
• Provide technical input for project planning, estimation, feasibilityassessment, and solution design.
• Develop optimal, scalable solutions and provide customer-facing technicalleadership throughout delivery.

Preferred skill sets:

Databricks | DatabricksAdministration | Azure | AWS | GCP | Apache Spark | PySpark | Python | SQL |Delta Lake | Unity Catalog

Years of experiencerequired:

9 to 13 Years

E/B.Tech/M.Tech/MBA/MCA