Senior Data Engineer

Mirai Tech Recruitment; Greater London; Hybrid; GBP 62,000 - 102,000; Full time

Job summary

We are recruiting for a large organisation for a senior hands-on contract role based in London. This Senior Data Engineer will design and improve engineering patterns across Azure Databricks and Azure Data Factory, drive performance, and act as a technical quality gate.

The role is London hybrid (one day onsite per week), 3-6 months, inside IR35, ASAP start. If you have strong SQL, Delta Lake, Unity Catalog and CI/CD experience, apply now.

Qualifications

• Strong hands-on experience across Azure Databricks.
• Strong hands-on experience across Azure Data Factory.
• Strong hands-on experience across Spark / PySpark.
• Advanced SQL and performance optimisation experience.
• Experience with CI/CD and Azure DevOps.
• Experience with Delta Lake and medallion architecture.
• Experience with Unity Catalog.
• Experience with data governance and orchestration.
• Experience with API-based data ingestion.
• Experience working with high-volume operational, IoT or telemetry data is particularly valuable.
• Experience at Senior or Lead Data Engineer level.
• Ability to work autonomously and hit the ground running.

Responsibilities

• Design and improve engineering patterns across Azure Databricks and Azure Data Factory.
• Drive performance and optimisation across caching, indexing, partitioning and cost.
• Review code and act as a technical quality gate.
• Improve DevOps practices across CI/CD, environment promotion, branching and release management.
• Identify opportunities to automate repetitive engineering processes.
• Work with advanced SQL across existing engineering patterns.
• Support data quality, governance and orchestration.
• Work closely with data engineering, analytics, data science and platform teams.
• Help establish consistent engineering standards across the wider service.
• Quickly understand an established data estate and identify opportunities for improvement.
• Provide immediate technical leadership across the data engineering function.

Job description

Salary: £62,000 - 102,000 per year

Requirements:

• Strong hands-on experience across Azure Databricks
• Strong hands-on experience across Azure Data Factory
• Strong hands-on experience across Spark / PySpark
• Advanced SQL and performance optimisation experience
• Experience with CI/CD and Azure DevOps
• Experience with Delta Lake and medallion architecture
• Experience with Unity Catalog
• Experience with data governance and orchestration
• Experience with API-based data ingestion
• Experience working with high-volume operational, IoT or telemetry data is particularly valuable
• Experience at Senior or Lead Data Engineer level
• Ability to work autonomously and hit the ground running

Responsibilities:

• Design and improve engineering patterns across Azure Databricks and Azure Data Factory
• Drive performance and optimisation across caching, indexing, partitioning and cost
• Review code and act as a technical quality gate
• Improve DevOps practices across CI/CD, environment promotion, branching and release management
• Identify opportunities to automate repetitive engineering processes
• Work with advanced SQL across existing engineering patterns
• Support data quality, governance and orchestration
• Work closely with data engineering, analytics, data science and platform teams
• Help establish consistent engineering standards across the wider service
• Quickly understand an established data estate and identify opportunities for improvement
• Provide immediate technical leadership across the data engineering function

Technologies:

• API
• Azure
• CI/CD
• Databricks
• DevOps
• IoT
• Support
• PySpark
• SQL
• Spark
• Unity
• Cloud
• GameDev

More:

We are a large, complex global organisation offering a senior hands‑on contract role for an experienced Senior Data Engineer. This is a London hybrid position with one day per week onsite, a 3-6 month duration, inside IR35, and an ASAP start. We are committed to creating an inclusive recruitment process and encourage applications from people of all backgrounds, experiences and identities. If this role sounds interesting but you do not meet every requirement, we would still encourage you to apply.

last updated 40 week of 2026