Data Engineer- Azure Databricks- Senior Associate

PwC Acceleration Center India

Bengaluru

On-site

INR 1,600,000 - 2,100,000

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Job summary

PwC Acceleration Centers in Bengaluru is seeking a Senior Associate in Data & Analytics Engineering to design and implement cloud data pipelines and analytics solutions. You will build scalable data infrastructure, mentor junior engineers, and collaborate with advisory and assurance teams on client engagements.

The role emphasizes strong SQL, data modelling, and experience with Hadoop, Spark, and Azure cloud services.

Qualifications

4+ years of experience in Azure ADLS, Data Bricks, Data Flows, HDInsight, Azure Analysis services.
Awareness of Data Management concepts and Data Modelling.
Good knowledge of Big Data frameworks and related technologies.
Experience in Hadoop and Spark is mandatory.
Expertise in extracting data from flat files, XML, JSON, Parquet, RDBMS, and other sources.
Clear understanding of DW lifecycle and related documentation.
Strong SQL with joins, groups, functions and stored procedures.
Experience in Unit and Integration Test creation and verification.
Documentation of Project Architecture, End User Manual and Operations Hand-off.

Responsibilities

Design and develop data solutions using Azure Databricks.
Collaborate with cross-functional teams to deliver analytics projects.
Analyze data to derive actionable insights for clients.
Enhance data processing workflows for efficiency.
Mentor junior data engineers and support knowledge transfer.
Stay updated on advancements in data engineering technologies.
Maintain adherence to data governance and security protocols.
Conduct training on best practices in data management.

Skills

SQL

Unix Shell Scripting

Data Modelling

Analytics

Communication

Analytical Skills

Quality Processes

Problem Solving

Cloud Migration

Education

Bachelor's Degree in Engineering / MCA / M.Sc / M.E / M.Tech / MBA

Tools

Azure Databricks

Hadoop

Spark

Git

CI/CD Frameworks

Jenkins

GitLab

Code Pipeline

Code Build

Code Commit

Azure ADLS

HDInsight

Azure Analysis Services

Job description

At PwC, our people in data and analytics engineering focus on leveraging advanced technologies and techniques to design and develop robust data solutions for clients. They play a crucial role in transforming raw data into actionable insights, enabling informed decision-making and driving business growth.

In data engineering at PwC, you will focus on designing and building data infrastructure and systems to enable efficient data processing and analysis. You will be responsible for developing and implementing data pipelines, data integration, and data transformation solutions.

Position Requirements

Must Have:

4+ years of experience working in Azure ADLS, Data Bricks, Data Flows, HDInsight, Azure Analysis services
Should be aware of Data Management concepts and Data Modelling
Good knowledge of Big Data frameworks and related technologies
Experience in Hadoop and Spark is mandatory
Should have expertise in extracting data from different source systems like flat files, XML sources, JSON, Parquet files, Big data appliances, RDBMS, etc.
Good knowledge of Big Data frameworks and related technologies
Experience in Hadoop and Spark is mandatory
Should have clear understanding of DW lifecycle and contributed in preparing
Detail Level Design document
Unit Test plans
Code review reports
Must have strong sql, understand joins, groups, functions & Stored procedures
Must have experience in Unit and Integration Test creation and verification
Proficient in creating documentation related to Project Architecture, Project end User Manual and Operations Hand off guide
Should have good understanding of Data Migration processes, methods and project lifecycle
Good in Unix Shell Scripting skills
Strong analytical skills and troubleshooting skills
Should be well versed with quality processes and implementation
Excellent communication skills
Strong understanding of Cloud data migration processes, methods and project lifecycle
Good knowledge of Application DevOps tools (Git, CI/CD Frameworks)
Experience in Jenkins or Gitlab with rich experience in source code management like Code Pipeline, Code Build and Code Commit

Desired Knowledge / Skills

Certifications in Azure / Databricks
Experience in building stream-processing systems, using solutions such as Storm or Spark
Streaming
Experience in Big Data ML toolkits, such as Mahout, SparkML, or H2O
Experience in Azure functions
Knowledge in Python

The Opportunity

When you join PwC Acceleration Centers (ACs), you step into a pivotal role focused on actively supporting various Acceleration Center services, from Advisory to Assurance, Tax and Business Services. In our innovative hubs, you’ll engage in challenging projects and provide distinctive services to support client engagements through enhanced quality and innovation. You’ll also participate in dynamic and digitally enabled training that is designed to grow your technical and professional skills.
As part of the Data and Analytics Engineering team you will lead the end-to-end implementation of innovative Cloud data engineering solutions. As a Senior Associate, you will analyze complex problems, mentor junior team members, and build meaningful client relationships while navigating the intricacies of the data landscape. This role offers the chance to deepen your technical skills and enhance your professional brand in a dynamic and collaborative environment.

Responsibilities