Senior Data Engineer - Real-Time & Batch Data Platform

Lloyds Banking Group; City of Westminster; On-site; GBP 90,000 - 130,000; Full time

Benefits offered by this job

A generous pension contribution up to

Annual bonus award

Share schemes

Discounted shopping

30 days holiday

Wellbeing initiatives

Parental leave policies

Job summary

Lloyds Banking Group is seeking a Senior Data Engineer to join the Data Platform team in the UK. You will design, build and operate scalable data infrastructure handling real-time and batch data, enabling analytics and ML workloads.

You will work closely with engineers, data scientists and business stakeholders to ensure reliability, performance and data quality across data warehouses and pipelines.

Qualifications

• 5+ years' experience as a Data Engineer in production environments
• Experience with streaming technologies such as Kafka, Pub/Sub or Kinesis
• Solid understanding of data warehousing principles including relational modelling, query optimisation and administration
• Strong programming experience with Python, Go and/or SQL
• Experience using dbt for data modelling and transformation
• Experience with Apache Airflow or similar orchestration tooling
• Proven experience designing and building production systems on GCP and/or AWS
• Strong understanding of event-driven architectures and distributed systems
• Experience deploying and supporting batch and real-time machine learning workloads
• Experience with distributed processing technologies such as Spark or Apache Beam
• Strong understanding of data quality principles and engineering best practices
• Experience optimising database performance and maintaining production data systems

Responsibilities

• Design, build and operate scalable data infrastructure for real-time and batch data
• Ensure reliability, performance and availability of data warehouse and pipelines
• Collaborate with engineers, data scientists and business stakeholders to promote best practices in data engineering

Skills

Python

Go

SQL

Tools

Kafka

Pub/Sub

Kinesis

dbt

Airflow

GCP

AWS

Spark

Apache Beam

Terraform

Kubernetes

Job description

Lloyds Banking Group is seeking a Senior Data Engineer to join the Data Platform team in the UK. You will design, build and operate scalable data infrastructure handling real-time and batch data, enabling analytics and ML workloads.

You will work closely with engineers, data scientists and business stakeholders to ensure reliability, performance and data quality across data warehouses and pipelines.