Senior Data Engineer

TalkTalk; Manchester; On-site; GBP 70,000 - 90,000; Full time

Job summary

A leading telecommunications company is seeking a Lead Data Engineer to manage a high-performing team within the Data Engineering department. This role requires over 8 years of experience in data engineering, expertise in Microsoft Azure, and strong competencies in Databricks, Scala, and PySpark. You will lead hands-on development, coach your team, and be involved in strategic planning while working in an innovative environment that offers ample growth opportunities.

Qualifications

• 8+ years of experience in data engineering and 1+ as a Lead Data Engineer.
• Proven expertise with Microsoft Azure.
• Strong proficiency in Databricks with hands-on Scala (Spark) and PySpark.
• Expertise in Databricks for large-scale data engineering and analytics workloads.

Responsibilities

• Lead hands-on development and review of a quality Python and SQL codebase.
• Input into planning and estimating work / delivery roadmap.
• Keep up to date with Databricks platform developments.
• Coach and mentor a distributed team of data engineers.

Skills

Data Engineering

Python

SQL

Microsoft Azure

Databricks

Scala

PySpark

Job description

12 Month Fixed Term Contract

The Team

The Data Engineering team is positioned in the Delivery business unit of the IT function within the Data & Automation department sitting alongside the Report Engineering, Data Governance, Data Migrations and Automation teams.

Data Engineering is a perfect blend of Data Engineers and QA Engineers that are renowned for their forward thinking and trail blazing. Exceptionally talented, supportive and personable, it’s an area that’s challenging, provides excellent growth opportunities and rewarding.

Role Expectations

• Leading by example, through hands‑on development and review of a quality Python and SQL codebase.
• Inputting into planning and estimating work / delivery roadmap.
• Keeping up to date with Databricks platform developments and maintaining tech currency of internal systems.
• Coaching and mentoring a distributed team of data engineers across multiple delivery projects.
• Managing relationships with stakeholders across operations, technology, and business teams to understand requirements and ensure.

Must Have

• 8+ years’ experience in data engineering and 1+ experience as Lead Data Engineer
• Proven expertise with Microsoft Azure
• Strong proficiency in Databricks with hands‑on Scala (Spark) and PySpark .
• Expertise in Databricks for large‑scale data engineering and analytics workloads. Design and implement scalable data architectures using Databricks Unity Catalog, Delta Lake, and Apache Spark
• Implement advanced Delta Lake features including time travel, vacuum operations, and Z‑ordering.
• Desirable : certification in Databricks Data Engineer and / or Azure Data Engineer are a plus.