Azure Data Engineer: Cloud Pipelines & Lakehouse

Somerset Bridge Group; West of England; Hybrid; GBP 53,000 - 67,000; Full time

Benefits offered by this job

Hybrid work

Annual leave 25+ days

Bonus

Pension

Flexi-time

Wellbeing app Wrkit

Life Assurance

Cycle to Work

Job summary

Somerset Bridge Group is modernising its data platform to a cloud-based ecosystem using Azure Databricks, Delta Lake and Power BI. You’ll join the Enterprise Data Team to design, build, and optimise scalable data solutions for Pricing, Underwriting, Claims, and more.

The role focuses on building data pipelines, governance, security, and both real-time and batch processing in a large Azure environment.

Qualifications

• Proven experience building scalable data pipelines on Azure Databricks and Azure Data Factory.
• Strong SQL and Python (PySpark) skills for data transformation.
• Experience with Delta Lake, Spark and real-time streaming.

Responsibilities

• Design, build and maintain scalable data pipelines using Azure Databricks and Delta Lake.
• Develop cloud data solutions in Azure with high performance and cost efficiency.
• Create lakehouse data models to support analytics and ML.
• Automate and monitor data workflows with Databricks Workflows and Delta Live Tables.
• Ensure data quality, governance and security controls.
• Collaborate with Pricing, Underwriting, Data Science and Technology teams.
• Drive cloud architecture improvements and compliance with Unity Catalog and Purview.

Skills

Azure Databricks

Azure Data Factory

Delta Lake

PySpark / Python

SQL

Data warehousing

Security & governance

CI/CD / DevOps

Azure Monitor

Stakeholder mgmt

Tools

Azure DevOps

GitHub Actions

Databricks Workflows

Delta Live Tables

Unity Catalog

Purview

Job description

Somerset Bridge Group is modernising its data platform to a cloud-based ecosystem using Azure Databricks, Delta Lake and Power BI. You’ll join the Enterprise Data Team to design, build, and optimise scalable data solutions for Pricing, Underwriting, Claims, and more.

The role focuses on building data pipelines, governance, security, and both real-time and batch processing in a large Azure environment.