Cloud Data Engineer — Azure Databricks & Delta Lake

Somerset Bridge Group; West of England; Híbrido; GBP 53.000 - 67.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Hybrid working – 2 days in the office,

25 days annual leave

Discretionary annual bonus

Pension scheme – 5% employee, 6%-emloy

Flexible working

Wrkit wellbeing app

Enhanced parental leave

Life Assurance

Car Insurance Discount

Cycle to Work Scheme

Descripción de la vacante

Somerset Bridge Group is modernising its data platform to support analytics, reporting and AI-driven initiatives across Pricing, Underwriting and Claims. You will design, build and optimise scalable data pipelines on Azure Databricks with Delta Lake and Azure Data Factory, while ensuring data quality and governance.

Joining the Enterprise Data Team, you will collaborate with Data Architects, Analysts and Technology colleagues to deliver secure, high-performance data solutions that power data-led

Formación

• Strong experience building data pipelines with Azure Data Factory, Databricks and Azure Data Lake.
• Advanced SQL and Python (PySpark) skills for data transformation, automation and optimisation.
• Hands-on experience with Delta Lake, Spark, Databricks Workflows and Structured Streaming.
• Knowledge of data warehousing, lakehouse architectures and dimensional modelling.

Responsabilidades

• Design, build and maintain scalable data pipelines using Azure Databricks, Azure Data Factory and Delta Lake to support both real-time and batch data processing.
• Develop and optimise cloud-based data solutions within Azure, ensuring high performance, scalability and cost efficiency.
• Create robust data models and lakehouse architectures to support analytics, reporting and machine learning initiatives.
• Automate and monitor data workflows using tools such as Databricks Workflows, Delta Live Tables and Azure Monitor.
• Ensure data quality, integrity and governance through effective controls, security standards and regulatory compliance.
• Collaborate with stakeholders across Pricing, Underwriting, Data Science and Technology to deliver data-driven business solutions.
• Implement data security and governance frameworks, including access controls, data lineage and sensitive data protection.
• Drive continuous improvement by adopting new technologies and best practices across the Azure and Databricks ecosystem.

Conocimientos

Azure Data Factory

Databricks

Azure Data Lake

SQL

Python (PySpark)

Delta Lake

Spark

CI/CD

Azure DevOps

GitHub Actions

Infrastructure as Code

Unity Catalog

Purview

Streaming

Kafka

Herramientas

Databricks Workflows

Delta Live Tables

Azure Monitor

Descripción del empleo

Somerset Bridge Group is modernising its data platform to support analytics, reporting and AI-driven initiatives across Pricing, Underwriting and Claims. You will design, build and optimise scalable data pipelines on Azure Databricks with Delta Lake and Azure Data Factory, while ensuring data quality and governance.

Joining the Enterprise Data Team, you will collaborate with Data Architects, Analysts and Technology colleagues to deliver secure, high-performance data solutions that power data-led