Principal Data Engineer

Epam; London; On-site; GBP 70,000 - 100,000; Full time

Job summary

A global leader in digital platform engineering is seeking a Principal Data Engineer in London. This role involves leading a team of data engineers, building and managing data pipelines, and collaborating with various stakeholders to ensure high-quality data management. The ideal candidate will possess extensive experience in Azure data technologies, a strong leadership background, and a passion for innovation. Join us to foster a dynamic, inclusive culture and grow your potential in a collaborative environment.

Benefits

Group pension plan

Private medical insurance

Employee assistance program

Learning and development opportunities

Gym discounts

Free lunches

Qualifications

• Minimum of 8 years of experience in data engineering.
• Proven leadership experience in managing data engineering teams.
• Proficiency in Azure data services and Python with Pandas.

Responsibilities

• Lead and manage a team of Azure data engineers.
• Design and implement scalable and secure data lake solutions.
• Build end-to-end data pipelines for enterprise-wide analytics.

Skills

Azure

Databricks

PySpark

Python

T-SQL

SparkSQL

Data modeling

ETL processes

Leadership

Communication

Education

Azure Data Engineer Associate certification

Tools

Git

Job description

Social network you want to login/join with:

Description

About the role

Are you a Principal Data Engineer with a passion for big data? Do you keep up to date with cutting-edge technologies within D&A?

If so, then you have a fantastic opportunity to join a multi-disciplinary team of engineers, architects, designers, and strategists as we continue to grow our Data & Analytics practice across Europe.
We're looking for a Principal Data Engineer with Azure, Databricks, and PySpark to join our team in London. The ideal candidate will have a strong background in data engineering, extensive experience with Azure cloud services, and experience leading a technical team on implementations.

Responsibilities

• Lead, mentor, and manage a team of Azure data engineers
• Drive the team's technical execution
• Collaborate with cross-functional teams including data scientists, analysts, and business stakeholders to ensure a quality single version of the truth
• Build end-to-end pipelines to support enterprise-wide analytics
• Design, develop, and implement scalable and secure data lake solutions on Azure
• Ensure best practices in data engineering, data integration, and ETL processes
• Code complex aspects of our pipeline as needed
• Maintain the maturity of our SVOT framework
• Monitor performance and scalability of the SVOT platform, updating the framework and code to ensure high availability and accessibility

Requirements

• Minimum of 8 years of experience in data engineering
• At least 5 years of hands-on experience with Azure data services (Apache Spark, Azure Data Factory, Synapse Analytics, RDBMS such as SQL Server)
• Proven leadership and management experience in data engineering teams
• Proficiency in PySpark, Python (with Pandas), T-SQL, SparkSQL, and experience with CI/CD pipelines
• Strong understanding of data modeling, ETL processes, and data warehousing concepts
• Knowledge of version control systems like Git
• Excellent problem-solving, analytical, communication, and collaboration skills
• Ability to manage multiple projects and meet deadlines
• Azure certifications such as Microsoft Certified: Azure Data Engineer Associate or Azure Solutions Architect

Nice to Have

• Experience with Scala for Apache Spark
• Knowledge of other cloud platforms like AWS or GCP

Our Benefits Include

• Group pension plan, life assurance, income protection, and critical illness cover
• Private medical insurance and dental care
• Cyclescheme, Techscheme, season ticket loans
• Employee assistance program
• Learning and development opportunities, certifications, well-being programs, and more
• EPAM Employee Stock Purchase Plan (ESPP)
• Perks such as gym discounts, free lunches, massages, and social events

About EPAM

• EPAM is a global leader in digital platform engineering and development services. We foster a dynamic, inclusive culture, collaborating with diverse teams on innovative projects. Join us to grow your potential and be part of a community that values your contributions.