Data Engineer, Azure Databricks Pipelines

Inigo; England; Hybrid; GBP 65,000 - 100,000; Full time

Job summary

Inigo is building a data platform on Azure to power underwriting insights, pricing and risk. We’re seeking a Data Engineer to own end-to-end pipelines in Databricks, orchestrated by Azure Data Factory, delivering curated datasets for analysts and actuaries.

You’ll write clean Python and SQL, model data with dimensional schemas, and collaborate with data scientists, engineers and business stakeholders. The role offers fast impact, a collaborative culture, and opportunities to grow into platform

Qualifications

• Experience building data pipelines in Python and writing clean, testable code.
• Strong SQL, including query optimisation and understanding of data modelling and storage.
• Practical experience of dimensional modelling—star schemas, facts and dimensions, defining grain.
• Hands-on experience with Spark, ideally PySpark on Databricks.
• Experience building and scheduling pipelines in Azure Data Factory or comparable tooling.
• Integrating with third-party REST APIs—token auth, pagination, rate limits.
• Working knowledge of core Azure data services (e.g., ADLS Gen2, Azure SQL Database).
• Familiarity with Git and CI/CD—code reviewed, merged and deployed through pipelines.
• Understanding ETL/ELT patterns and impact on pipelines.
• Ability to translate business problems into well-designed data solutions.

Responsibilities

• Building, testing and maintaining data pipelines in Databricks and Python.
• Orchestrating and scheduling pipelines in Azure Data Factory.
• Modelling curated datasets with clear grain and history.
• Writing and optimising SQL against Delta Lake and the platform.
• Ingesting data from files, APIs and databases with incremental loads.
• Deploying work safely to production using CI/CD pipelines and automated tests.
• Monitoring pipelines in production and fixing failures.
• Collaborating with analysts, data scientists and actuaries to translate problems into solutions.
• Contributing to code reviews and keeping code and docs up to date.

Skills

Python

SQL

Dimensional modelling

PySpark

Data pipelines

Tools

Databricks

Azure Data Factory

Power BI

Delta Lake

Git / CI/CD

Azure SQL Database

Azure Data Lake Storage Gen2

Job description

Inigo is building a data platform on Azure to power underwriting insights, pricing and risk. We’re seeking a Data Engineer to own end-to-end pipelines in Databricks, orchestrated by Azure Data Factory, delivering curated datasets for analysts and actuaries.

You’ll write clean Python and SQL, model data with dimensional schemas, and collaborate with data scientists, engineers and business stakeholders. The role offers fast impact, a collaborative culture, and opportunities to grow into platform