Senior Data Engineer - Scalable AI Data Platform

The BDPN

Greater London

Hybrid

GBP 110,000 - 160,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

The BDPN is seeking a Principal Data Engineer to drive the technical direction of our data platform across ingestion, transformation, storage and governance. You will design and implement scalable pipelines that process large data volumes in real-time and batch, enabling stakeholders to build advanced analytics and AI features.

You will lead architecture and deployment of Big Data systems, evaluate new tools, and mentor engineering teams while partnering with analytics, product and platform

Qualifications

  • Experience building large-scale data platforms in production.
  • Strong knowledge of distributed computing and data architecture.
  • Hands-on with ETL/ELT pipelines and data governance.

Responsibilities

  • Lead data architecture and deployment of scalable Big Data systems.
  • Develop end-to-end ETL/ELT pipelines and data ingestion frameworks.
  • Design data lakes, warehouses, and real-time streaming architectures.
  • Collaborate with analytics, product, and platform teams on data models.
  • Implement data quality, governance, and observability at scale.

Skills

Data engineering
Python
SQL
Cloud platforms

Tools

Apache Spark
Flink
Hadoop
Kafka
Kinesis
Airflow
Dagster
Snowflake
BigQuery
Redshift
Databricks
Delta Lake
Apache Iceberg

Job description

The BDPN is seeking a Principal Data Engineer to drive the technical direction of our data platform across ingestion, transformation, storage and governance. You will design and implement scalable pipelines that process large data volumes in real-time and batch, enabling stakeholders to build advanced analytics and AI features.

You will lead architecture and deployment of Big Data systems, evaluate new tools, and mentor engineering teams while partnering with analytics, product and platform

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Principal Data Engineer - Cloud & Data Platforms
Principal Data Engineer - Cloud & Data Platforms

Boston Consulting Group • Greater London

On-site
GBP 100,000 - 180,000
Senior Data Platform Engineer - Lead for AI & Cloud
Senior Data Platform Engineer - Lead for AI & Cloud

StepStone Group • United Kingdom

Hybrid
GBP 100,000 - 150,000
29 days holiday
Private medical
Dental healthcare
+7
Data Platform Lead: Scalable, AI‑Driven Data Solutions
Data Platform Lead: Scalable, AI‑Driven Data Solutions

PA Consulting Group • City Of London

Hybrid
GBP 90,000 - 140,000
Health and lifestyle perks
25 days annual leave (+ Xmas Eve half‑
Generous pension
+3
Senior AI Architect: Build Scalable Data & AI Platforms
Senior AI Architect: Build Scalable Data & AI Platforms

Datatonic • Greater London

Hybrid
GBP 120,000 - 180,000
Holiday: 25 days + bank holidays
Private health insurance
Gym membership discounts
+7
Senior Data Engineer - Build AI-Ready Data Platforms
Senior Data Engineer - Build AI-Ready Data Platforms

And Digital • West of England

On-site
GBP 75,000 - 110,000
25 days holiday
Pension: 6% + 2%
Healthcare cover
+5
Senior Data Engineer — Lead Data Teams, Hybrid Role
Senior Data Engineer — Lead Data Teams, Hybrid Role

Bae • England

Hybrid
GBP 70,000 - 100,000
Private medical and dental insurance
Pension scheme
Cycle to work scheme
+3
Remote Analytics Engineer - Data Infrastructure for AI
Remote Analytics Engineer - Data Infrastructure for AI

Prolific • Greater London

Remote
GBP 75,000 - 110,000
Remote work
Competitive salary
Benefits
Senior Data Engineer — AI-Driven ETL & Data Pipelines
Senior Data Engineer — AI-Driven ETL & Data Pipelines

mnAi • Crawley

Hybrid
GBP 60,000 - 80,000
Hybrid working options
Ownership of production data systems
Access to AI development tools
+1
Lead Data Engineer — AI Data Platform & Pipelines
Lead Data Engineer — AI Data Platform & Pipelines

Resume Skip • Greater London

On-site
GBP 90,000 - 130,000
Unlimited leaves
Life insurance & private health
No hierarchy
+1
Data Engineering Lead: AI-Driven Platforms & Leadership
Data Engineering Lead: AI-Driven Platforms & Leadership

NewDay • Greater London

Hybrid
GBP 90,000 - 140,000