Senior Data Engineer

IO Associates; United Kingdom; À distance; GBP 90 000 - 130 000; Plein temps

Avantages offerts par ce poste

Remote work

Outside IR35

12-month engagement

Résumé du poste

IO Associates is hiring a Senior Data Engineer in the United Kingdom to lead data engineering delivery on Azure and Databricks. You will own technical direction across multiple projects, mentor engineers, and collaborate with governance, migrations and automation teams.

You will design scalable pipelines, migrate data, and implement Delta Lake features while ensuring quality, reliability and performance. This is a remote, outside IR35, 12-month engagement.

Qualifications

• 8+ years of data engineering with leadership experience.
• Strong Microsoft Azure and Databricks expertise.
• Proven experience delivering large-scale data projects.

Responsabilités

• Design scalable pipelines and lakehouse architectures on Databricks.
• Move data from legacy systems to the new platform with quality focus.
• Write production-grade Python and SQL for multiple projects.
• Develop Databricks integrations with external systems.
• Apply advanced Delta Lake features to control performance and cost.
• Support planning, estimation, and delivery roadmaps.
• Mentor distributed data engineering teams across workstreams.
• Collaborate with stakeholders to deliver data products.

Connaissances

Azure

Databricks

Python

SQL

Leadership

Data migration

Formation

Databricks/Azure Data Engineer certification

Outils

Delta Lake

Unity Catalog

Azure Databricks

Description du poste

United Kingdom

Senior Data Engineer

Posted on Monday, 12th October 2026

Our client, a leading UK telecommunications provider, is running a programme of data engineering delivery on Azure and Databricks. The work covers data migration, legacy system modernisation and the ongoing build-out of a Delta Lake-based platform. Delivery is spread across several concurrent projects, and the client needs a senior engineer who can own the technical direction on them.

You will join a Data Engineering team of developers and QA engineers inside IT Delivery. The team works alongside Report Engineering, Data Governance, Data Migrations and Automation, so you will be building and handing over work across all of these functions.

What you’ll deliver

• Design and build scalable pipelines and lakehouse architecture on Databricks, using Unity Catalog, Delta Lake and Apache Spark
• Move data from legacy systems into the new platform, with a focus on quality and reliability
• Write and review production-grade Python and SQL, setting the engineering standard across projects
• Build custom solutions using Databricks APIs and integrate them with external systems
• Apply advanced Delta Lake features (time travel, vacuum operations, Z-ordering) to keep performance and cost under control
• Support planning and estimation, and help shape the delivery roadmap
• Mentor a distributed team of data engineers across multiple workstreams
• Work with operations, technology and business stakeholders to turn requirements into delivered data products

What you’ll bring

• 8+ years in data engineering, including at least 1 year in a lead capacity
• Proven Microsoft Azure expertise
• Strong, hands‑on Databricks experience with Scala (Spark) and PySpark
• Experience delivering large‑scale data engineering and analytics workloads on Databricks
• A track record on data migration or legacy modernisation projects
• Databricks Data Engineer or Azure Data Engineer certification (desirable)

Why this project

• A 12-month engagement with a clear delivery scope
• Senior ownership of technical decisions across several live projects
• Remote
• Outside IR35