Databricks Lead Engineer | Senior Manager | Data & AI | Advisory

Dormont Manufacturing Co; Dublin; On-site; EUR 75,000 - 95,000; Full time

Benefits offered by this job

Competitive compensation package

Flexible work programs

Inclusive employee benefits

Job summary

PwC Ireland is looking for a Lead Developer in the Data & AI team, focusing on enhancing healthcare outcomes through analytics projects. You will be responsible for developing data solutions using Databricks technology and coordinating with a skilled team of architects, engineers, and developers.

The ideal candidate has over 5 years of experience in data engineering and expertise in technologies like PySpark, Delta Lake, and SQL. PwC values innovation and teamwork to make impactful changes in the healthcare sector.

Qualifications

• Over 5 years of experience in data engineering and software engineering.
• Demonstrated experience in leading projects and mentoring teams.
• Expertise in managing stakeholders and resolving complex problems.

Responsibilities

• Develop and optimize complex data pipelines using PySpark/Scala and SQL.
• Guide development teams and share best practices in a mentorship role.
• Implement data quality checks and work on cloud services to manage Databricks clusters.

Skills

Deep Spark (PySpark/Scala)

Delta Lake

SQL

Databricks Notebooks/Workflows

Cloud Platforms (Azure/AWS)

CI/CD tools

Job description

Line of Service

Advisory

Industry/Sector

Not Applicable

Specialism

Technology Strategy

Management Level

Senior Manager

Job Description & Summary

At PwC, our people in data and analytics focus on leveraging data to drive insights and make informed business decisions. They utilise advanced analytics techniques to help clients optimise their operations and achieve their strategic goals.

In data analysis at PwC, you will focus on utilising advanced analytical techniques to extract insights from large datasets and drive data-driven decision-making. You will leverage skills in data manipulation, visualisation, and statistical modelling to support clients in solving complex business problems.

PwC Ireland- Bring Your Talent, Grow Your Skills, Unlock Your Potential

Do you want to be a part of a team of technology and digital leaders, experienced in scoping, and delivering large transformation programmes in complex matrix environments? Our Data & AI Consulting team is dedicated to leveraging technology in order to solve complex business problems for our clients. We work with Ireland’s largest organisations across a wide range of industries including financial services, government, healthcare, technology, media and telecoms.

Do you want to be a part of a community of problem solvers combining human ingenuity, experience and technology innovation to deliver sustained outcomes and build trust?

At PwC, we believe real transformation is Human Led and Tech Powered. Key to delivering this capability and driving our continued growth is our ability to guide clients in their consideration and implementation of leading Data, Analytics & AI Technologies. As we embark on further building on our deep technical and commercial teams, we are hiring for talent with skills in data strategy, governance, and transformation. By joining our Data & AI team you will have the opportunity to help shape the future of our company and help clients deliver bigger, more impactful transformation that lasts.

Purpose-led work you’ll be part of

As the Lead Developer on our Data & AI team, you will spearhead the development and delivery of analytics projects that significantly enhance healthcare outcomes. Your role will be pivotal in developing, leading, reviewing, and managing code with a focus on Databricks technology for complex healthcare systems. Ensuring the highest standards of code development, delivery, data quality, compliance, and reliability will be a core part of your responsibilities. Collaborating with experienced architects, engineers, and junior developers, you will contribute to building a robust data solution. Your efforts will directly support improved decision-making for healthcare providers, patients, and policymakers. By leveraging your expertise in healthcare data standards and best practices, you will be instrumental in advancing projects that positively impact the sector.

Role & Responsibilities

• Architecture & Design: Developing scalable data solutions, defining data models, designing data lakes/lakehouses, and creating high-level design documents.
• Development & Optimization: Building and optimizing complex data pipelines (both batch and streaming) using PySpark/Scala, Delta Lake, and SQL, as well as automating workflows.
• Leadership & Mentoring: Guiding development teams, sharing best practices, mentoring engineers, and leading technical initiatives.
• Data Governance & Quality: Implementing data quality checks, security measures (RBAC, UnityCatalog), and ensuring data lineage.
• Cloud & DevOps: Working with cloud services (Azure/AWS), implementing CI/CD pipelines (Git, ADO), and managing Databricks clusters.
• Collaboration: Partnering with data scientists, analysts, and business stakeholders to translate requirements into technical solutions.

Your Profile

• Technical Skills: Expertise in Deep Spark (PySpark/Scala), Delta Lake, UnityCatalog, SQL, Databricks Notebooks/Workflows, Cloud Platforms (Azure/AWS), and CI/CD tools, along with proficiency in data modeling.
• Leadership Abilities: Demonstrated experience in leading projects, mentoring teams, architectural design, managing stakeholders, and resolving complex problems.
• Professional Experience: Over 5 years of experience in data engineering and software engineering, with substantial exposure to Databricks.

Unlock your potential with PwC Ireland

We believe that challenges are better solved together! We’re inspiring and empowering our people to change the world. Powered by the latest technology, you’ll be a part of amazing teams encouraged to collaborate and innovate in a way that creates powerful solutions and makes a positive impact. This purpose-led work, and our continuous development and encouragement, will help unlock your potential and take your career to the next level.

Enjoy PwC’s perks

We reward your impact, and support your wellbeing, through a competitive compensation package, inclusive employee benefits and flexibility programs that will help you thrive in work and life. Learn more about us at Life@PwC. https://www.pwc.ie/careers-ie/life-at-pwc.html.

Being appreciated for being you

Our most valuable asset is our people and we grow stronger as we learn from one another. We are an equal opportunity employer and we value diversity. We do not discriminate on the basis of race, religion, colour, national origin, sex, gender, gender expression, sexual orientation, age, marital status, veteran status, or disability status. We believe this so much that we have signed up for Business in the Community Elevate Pledge (https://www.bitc.ie/the-leaders-group-on-sustainability/inclusive-workplace-pledge/).

You can learn more about our culture of belonging and explore our range of inclusive programmes, initiatives, employee resource groups and more at www.pwc.ie.

Avoid the confidence gap; you do not have to match all the listed requirements exactly to apply, we’d love to hear from you!

We will ensure that individuals with disabilities are provided reasonable accommodation to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment. Please email us for more information.

Education (if blank, degree and/or field of study not specified)

Degrees/Field of Study required:

Degrees/Field of Study preferred:

Certifications (if blank, certifications not specified)

Required Skills

Optional Skills

Accepting Feedback, Accepting Feedback, Active Listening, Algorithm Development, Alteryx (Automation Platform), Analytical Thinking, Analytic Research, Big Data, Business Data Analytics, Coaching and Feedback, Communication, Complex Data Analysis, Conducting Research, Creativity, Customer Analysis, Customer Needs Analysis, Dashboard Creation, Data Analysis, Data Analysis Software, Data Collection, Data-Driven Insights, Data Integration, Data Integrity, Data Mining, Data Modeling {+ 46 more}

Desired Languages (If blank, desired languages not specified)

Travel Requirements

Not Specified

Available for Work Visa Sponsorship?

Yes

Government Clearance Required?

No

Job Posting End Date