Senior Data Engineer: AWS Data Platform & ML Pipelines

Understanding Recruitment Ltd; Greater London; On-site; GBP 85,000 - 120,000; Full time

Job summary

Understanding Recruitment Ltd is hiring a Senior Data Engineer to take ownership of a modern, AWS-native data platform in London. You'll shape data ingestion, transformation and scalable pipelines powering production ML and AI initiatives.

This hands-on role sits in a lean engineering team, collaborating with data scientists, ML engineers and architects to ensure reliable, high-quality data across batch and streaming workloads, and to evolve the data lake architecture.

Qualifications

• Hands-on experience building production data engineering systems.
• Deep familiarity with AWS and cloud-native data architectures.
• Strong Python and SQL skills with solid software engineering fundamentals.
• Experience designing and maintaining scalable ETL/ELT pipelines and data lakes.
• Understanding of distributed data processing and streaming architectures.
• Ability to take end-to-end ownership and solve technical problems autonomously.
• Experience with Apache Iceberg, Kafka, Parquet or Athena is highly desirable; SageMaker exposure is a bonus.

Responsibilities

• Design, build and maintain scalable, production-grade data pipelines across batch and streaming workloads.
• Evolve the AWS data lake architecture to improve data quality, reliability and performance.
• Build and optimise data ingestion, transformation and processing workflows with modern tools.
• Collaborate with data scientists, ML engineers and architects to ensure high-quality data for production ML systems.
• Influence architectural decisions and establish engineering best practices for the data platform.

Skills

Python

SQL

Data engineering

End-to-end ownership

Tools

Apache Iceberg

Kafka

Parquet

Athena

SageMaker

Job description

Understanding Recruitment Ltd is hiring a Senior Data Engineer to take ownership of a modern, AWS-native data platform in London. You'll shape data ingestion, transformation and scalable pipelines powering production ML and AI initiatives.

This hands-on role sits in a lean engineering team, collaborating with data scientists, ML engineers and architects to ensure reliable, high-quality data across batch and streaming workloads, and to evolve the data lake architecture.