Senior Data Engineer — Databricks Platform & Governance

Stack Infrastructure Emea; Greater London; On-site; GBP 90,000 - 130,000; Full time

Job summary

Stack Infrastructure Emea in London is seeking a Senior Data Engineer to build and manage Stack’s data platform in Databricks, creating reliable pipelines from multiple source systems and ensuring governance and security for business insights.

You will collaborate with Finance, Delivery, Operations and HR to improve reporting, implement data quality checks and support decision-making while optimizing costs and performance.

Qualifications

• Significant experience as a Data Engineer, including ownership of production data pipelines.
• Strong hands-on experience with Databricks, Delta Lake, Python, SQL and PySpark.
• Experience integrating data from APIs, business applications, databases and files.
• Experience with Unity Catalog, access management, data governance and data quality controls.

Responsibilities

• Design, build and maintain data pipelines from APIs, databases, SaaS applications, files and spreadsheets into Databricks.
• Develop scalable data models and Delta Lake architecture for reporting, analytics and future AI use cases.
• Manage Databricks Workflows, Lakeflow or Delta Live Tables, SQL Warehouses and related processing jobs.
• Administer Databricks workspaces, Unity Catalog, permissions, service principals.

Skills

Databricks

Delta Lake

PySpark

Python

SQL

Unity Catalog

Data governance

Data quality checks

CI/CD

Power BI

Production data pipelines

Education

Databricks Certification

Tools

Databricks

Delta Lake

SQL Warehouses

Unity Catalog

Git

Azure Data Services

Power BI

CI/CD

Job description

Stack Infrastructure Emea in London is seeking a Senior Data Engineer to build and manage Stack’s data platform in Databricks, creating reliable pipelines from multiple source systems and ensuring governance and security for business insights.

You will collaborate with Finance, Delivery, Operations and HR to improve reporting, implement data quality checks and support decision-making while optimizing costs and performance.