Lead Data Engineer: AWS Lakehouse & Real-Time Streaming

Lorien; Manchester; Hybride; GBP 85 000 - 100 000; Plein temps

Résumé du poste

Lorien in Manchester is seeking a Lead Data Engineer who stays hands-on, sets the standard and leads by example. You will own the data platform migration to an AWS lakehouse and shape the real-time streaming architecture to slash report latency from hours to minutes.

As the senior technical authority, you’ll guide architecture, review code, mentor engineers, and partner with data scientists, analysts and senior stakeholders. This is a hybrid role with two days per week in central Manchester.

Qualifications

• Lead on a large-scale data platform or data warehouse migration.
• Hands-on AWS data stack including Glue, Lambda, S3, Redshift and EMR.
• Advanced Python, PySpark and SQL capabilities.
• Production experience with Kafka or MSK and Spark Streaming.
• Experience with Iceberg or Delta Lake data formats.
• Strong data modelling and lakehouse design.
• Proven ability to mentor engineers and set standards.

Responsabilités

• Act as the senior technical authority for the data platform.
• Drive migration from on-prem SQL Server to AWS lakehouse architecture.
• Design real-time streaming to reduce report latency to minutes.
• Set standards, review code, and guide design decisions.
• Collaborate with data scientists, analysts and other engineering leads.
• Communicate effectively with senior stakeholders and manage expectations.

Connaissances

Technical lead experience

AWS data experience

Python

PySpark

SQL

Data modelling

Terraform

CI/CD

Mentoring engineers

Stakeholder communication

Spark Structured Streaming

Apache Iceberg

Delta Lake

Kafka / MSK

Outils

AWS Glue

AWS Lambda

Amazon S3

Amazon Redshift

AWS EMR

Kafka / MSK

Spark Structured Streaming

Iceberg

Delta Lake

Terraform

CI/CD pipelines

Description du poste

Lorien in Manchester is seeking a Lead Data Engineer who stays hands-on, sets the standard and leads by example. You will own the data platform migration to an AWS lakehouse and shape the real-time streaming architecture to slash report latency from hours to minutes.

As the senior technical authority, you’ll guide architecture, review code, mentor engineers, and partner with data scientists, analysts and senior stakeholders. This is a hybrid role with two days per week in central Manchester.