Data Platform Engineer - Databricks & Pipelines (London)

Barclays; Greater London; Hybrid; GBP 90,000 - 130,000; Full time

Benefits offered by this job

Hybrid work pattern

Job summary

Barclays is seeking a Data Engineer based in London to build and maintain data pipelines, data lakes and warehouses on Databricks. You will work with Python, PySpark and SQL to deliver production-grade platforms and data products that domain teams can reuse.

You will help establish engineering standards, automation and secure-by-design practices, while moving toward self-service tooling for data products across the organization. This is a hybrid role with 3 days in the office.

Qualifications

• Hands-on experience building production-grade Databricks platforms and reliable batch and streaming pipelines using Python, PySpark and SQL.
• Modular, maintainable components with automated pipeline testing, Git-based workflows and CI/CD deployment.
• Experience engineering systems for sensitive or regulated data, including IAM, encryption, secrets management and monitoring.

Responsibilities

• Build and maintain data architectures, pipelines and data lakes to ensure accurate, accessible and secure data.
• Design data warehouses and data lakes with appropriate security measures and data volumes.
• Develop processing and analysis algorithms for complex data.
• Collaborate with data scientists to build and deploy machine learning models.

Skills

Databricks

Data pipelines

Python

PySpark

SQL

CI/CD

Security controls

Unity Catalog

Terraform

Databricks Asset Bundles

GitHub Actions

Platform self-service

Tools

Terraform

Databricks Asset Bundles

GitHub Actions

Job description

Barclays is seeking a Data Engineer based in London to build and maintain data pipelines, data lakes and warehouses on Databricks. You will work with Python, PySpark and SQL to deliver production-grade platforms and data products that domain teams can reuse.

You will help establish engineering standards, automation and secure-by-design practices, while moving toward self-service tooling for data products across the organization. This is a hybrid role with 3 days in the office.