Data Engineering Manager with Azure & Databricks

PwC; Gurugram District, Bengaluru, Hyderabad; On-site; INR 4,200,000 - 6,800,000; Full time

Job summary

PwC in India is seeking a senior data platform architect to lead the design and delivery of Azure-based data platforms, including ADLS, Databricks, Data Factory, and Synapse.

You will drive architecture, standards, and best practices for data modeling, data management, and governance while overseeing end-to-end migration projects and production handoffs.

Qualifications

• Experience leading teams in Azure-based data platforms.
• Ability to define standards for data modeling and data management.
• Hands-on and architectural experience with Spark and Hadoop ecosystems.
• Proven track record delivering data migration projects and end-to-end delivery.

Responsibilities

• Lead design and delivery of Azure-based data platforms with ADLS, Databricks, Data Factory, and Synapse.
• Define robust, scalable, and secure data architectures and standards.
• Manage end-to-end data migration projects with risk and scope oversight.
• Develop advanced SQL and data transformations; automate with Python and shell scripts.
• Guide DevOps practices to automate pipelines and ensure quality code delivery.
• Coordinate onsite/offshore teams and mentor junior engineers.
• Set up KPI-based data quality frameworks and production handoffs.

Skills

Technical leadership

Architectural design

Cloud proficiency

Data lifecycle ownership

Delivery management

Data engineering

DevOps & CI/CD

Stakeholder management

Operational excellence

Tools

Git

Jenkins

GitLab

Azure DevOps

Python

Unix/Shell

Job description

Role & responsibilities

• Technical Leadership: Extensive experience leading teams in the design and delivery of Azure-based data platforms utilizing ADLS, Databricks, Data Factory, and Synapse.
• Architectural Expertise: Demonstrated ability to design robust, scalable, and secure data architectures. You should be capable of defining technical standards and best practices for data modeling and data management.
• Cloud Proficiency: Advanced understanding of Big Data frameworks; deep hands-on and architectural experience with Spark and Hadoop ecosystems.
• Data Lifecycle Ownership: Comprehensive knowledge of the Data Warehousing (DW) lifecycle. Proven experience in leading the creation of high-level and detail-level design documents, and defining strategy for Unit/Integration/UAT plans.
• Delivery Management: Strong track record of owning end-to-end delivery of data migration projects, including scope management, risk mitigation, and resource planning.
• Technical Proficiency: Mastery of advanced SQL, complex data transformations, and performance tuning. Proficiency in Python and Unix/Shell scripting for automation.
• DevOps & CI/CD: Strategic understanding of DevOps implementation (Git, Jenkins, GitLab, Azure DevOps) to automate pipelines and ensure high-quality code delivery.
• Stakeholder & Team Management: Excellent communication skills with the ability to advise senior client stakeholders, manage onsite/offshore teams, and mentor junior associates/engineers.
• Operational Excellence: Experience in setting up operational frameworks, defining KPIs for data quality, and leading the transition to production (Operations Hand-off).

Good to Have

• Certifications: Azure Data Engineer (DP-203) or Databricks Certified Data Engineer Professional.
• Advanced Analytics: Knowledge of Big Data ML toolkits (SparkML, H2O) and experience building real-time stream-processing systems.
• Serverless Architectures: Practical experience with Azure Functions and event-driven data architectures.
• Strategic Consulting: Experience in pre-sales support, RFP responses, or presenting technical roadmaps to C-level or Director-level business stakeholders.

Preferred candidate profile