Senior Data Engineer | Azure Databricks, Python & SQL

Arthur Recruitment; Greater London; Hybrid; GBP 90,000 - 115,000; Full time

Benefits offered by this job

Bonus

Benefits

Job summary

Arthur Recruitment is helping an established London-based insurer recruit a hands-on Data Engineer to join the Data & Technology function. The role focuses on Databricks, Python and SQL to own data engineering solutions and deliver MI, analytics and reporting for underwriting and risk teams.

The candidate will build and optimize ETL/ELT pipelines, design data models and ensure data quality and governance across the organisation, with hybrid working in London.

Qualifications

• Strong Databricks, Python and SQL development experience.
• Hands-on ETL/ELT pipeline building experience.
• Insurance sector experience preferred (London Market/Lloyd's).

Responsibilities

• Develop, maintain and optimise ETL/ELT pipelines and data warehousing solutions.
• Work extensively with Databricks, Python and SQL to transform, process and manage data.
• Design and enhance data models to support reporting and analytics.
• Monitor and troubleshoot data pipelines, ensuring reliability, accuracy and performance.
• Work closely with Exposure Management and wider business teams to understand data requirements.
• Support the delivery of Management Information (MI), reporting and analytics solutions.
• Improve data quality, integration and governance across the business.
• Contribute to the ongoing development of the organisation's data engineering capabilities.

Skills

Databricks

Python

SQL

ETL/ELT pipelines

Data warehousing

Data modelling

Data governance

Job description

Location: London | Hybrid Working

Salary: 90k + 15% Bonus + benefits

Sector: London Insurance Market

I'm currently working with an established insurance business looking to hire a Data Engineer to join its growing Data & Technology function.

This is an excellent opportunity for a hands‑on Data Engineer with strong Databricks, Python and SQL experience who wants to take ownership of data engineering solutions whilst gaining valuable exposure to the wider insurance business, particularly Exposure Management, Data Analytics and MI.

Key Responsibilities

• Develop, maintain and optimise ETL/ELT pipelines and data warehousing solutions.
• Work extensively with Databricks, Python and SQL to transform, process and manage data.
• Design and enhance data models to support reporting and analytics.
• Monitor and troubleshoot data pipelines, ensuring reliability, accuracy and performance.
• Work closely with Exposure Management and wider business teams to understand data requirements.
• Support the delivery of Management Information (MI), reporting and analytics solutions.
• Improve data quality, integration and governance across the business.
• Contribute to the ongoing development of the organisation's data engineering capabilities.

What We're Looking For

• Strong commercial experience with Databricks, this is a key requirement.
• Excellent Python and SQL development skills.
• Experience building and maintaining ETL/ELT pipelines.
• Good understanding of data warehousing, data modelling and data integration.
• Experience troubleshooting and optimising data processing solutions.
• Ability to work collaboratively with technical teams and business stakeholders.
• Previous experience within insurance, ideally the London Market or Lloyd's.
• Exposure to underwriting, risk or Exposure Management data.
• Experience supporting MI, BI or analytical reporting solutions.
• Understanding of data quality, governance and regulatory requirements.

Why Consider This Opportunity?

• Hands‑on Databricks engineering: Work with Databricks, Python and SQL on business‑critical data solutions.
• Exposure to the insurance market: Work closely with Exposure Management teams and develop valuable insurance domain knowledge.
• Varied responsibilities: Opportunity to contribute across Data Engineering, Data Warehousing, Analytics and MI.
• Career development: Build broader technical and business expertise in a specialist insurance environment.
• Business impact: See how your engineering work directly supports underwriting, risk analysis and business decisions.