Data Platform Engineer: Pipelines, Lakes & Self‑Service

Barclays; City Of London; Hybrid; GBP 80,000 - 110,000; Full time

Job summary

Barclays is seeking a Data Engineer in London to build and maintain data pipelines, data warehouses and data lakes with a security-first mindset. You will collaborate with data scientists to deploy machine learning models and design scalable data platforms using Databricks on AWS, adopting IaC and CI/CD practices.

You will work in a hybrid setup with office presence in London, contributing to a high-scale payments data platform and guiding domain teams with reusable, self-service data products.

Qualifications

• Hands-on with production-grade Databricks platforms and batch/streaming pipelines (Python, PySpark, SQL).
• CI/CD pipelines and automated testing using GitHub Actions or similar.
• Security-focused data engineering including IAM, encryption, secrets management, least privilege.
• Unity Catalog configuration for lineage, permissions, metadata and governance.
• IaC with Terraform and Databricks deployment automation.
• Platform product mindset enabling self-service data products.

Responsibilities

• Build and maintain data pipelines, ensuring durable, complete, and consistent data.
• Design data warehouses and data lakes with appropriate scale and security.
• Develop processing and analysis algorithms for complex data.
• Collaborate with data scientists to deploy machine learning models.

Skills

Databricks

Python

PySpark

SQL

CI/CD

Terraform

GitHub Actions

Identity & Access Control

Encryption

Unity Catalog

Tools

Databricks

Terraform

GitHub Actions

Unity Catalog

Job description

Barclays is seeking a Data Engineer in London to build and maintain data pipelines, data warehouses and data lakes with a security-first mindset. You will collaborate with data scientists to deploy machine learning models and design scalable data platforms using Databricks on AWS, adopting IaC and CI/CD practices.

You will work in a hybrid setup with office presence in London, contributing to a high-scale payments data platform and guiding domain teams with reusable, self-service data products.