Senior Data Engineer - Azure AI & Lakehouse (Hybrid UK)

Intelligent Talent; Greater London; On-site; GBP 65,000 - 75,000; Full time

Benefits offered by this job

Annual bonus 10%

Private medical insurance

Employer pension 7%

Income protection

Job summary

Intelligent Talent in London (UK) seeks a Senior Data Engineer to evolve an Azure-based data platform, with a mix of greenfield work and enhancements to an established system. The team embraces AI tools like Copilot and Azure AI Foundry, and the role requires hands-on data engineering across pipelines, data models and integrations.

The ideal candidate has strong Python/PySpark and SQL skills, solid data governance understanding, and the ability to mentor peers in a security-conscious, regulated

Qualifications

• Hands-on Microsoft Fabric experience essential, ideally 1+ year supporting and evolving what you built after go-live.
• Strong Python/PySpark and advanced SQL, with solid data modelling, ETL/ELT, orchestration and data quality experience.
• Working understanding of data governance, privacy and security, with evidence of controls in what you build.
• Experience mentoring or giving technical guidance to other data engineers.
• Clear communication with technical and non-technical colleagues, and self-motivation to own outcomes.
• Working familiarity with AI tools such as Copilot or Azure AI Foundry; basic AI competence expected across the business.
• Bonus: financial services, wealth management or other regulated-sector experience is a real advantage.

Responsibilities

• Designing and building pipelines, data models and integrations across Lakehouse, Warehouse, OneLake, pipelines and notebooks, using Python, PySpark and advanced SQL.
• Connecting internal and third-party systems through APIs and SFTP.
• Automated testing, validation and data quality controls.
• Tuning performance, reliability and cloud cost.
• Taking production incidents through to root cause and lasting fix.
• Improving observability, resilience and maintainability across the platform.
• Git, Azure DevOps, CI/CD and code review in two-week Scrum sprints.
• Documenting designs and controls to governance, privacy and security standards of a regulated business.

Skills

Python

PySpark

Advanced SQL

Data modelling

ETL/ELT

Data governance

AI tools

Mentoring

Communication

Tools

Lakehouse

OneLake

Git

Azure DevOps

CI/CD

Microsoft Fabric

Azure AI Foundry

Job description

Intelligent Talent in London (UK) seeks a Senior Data Engineer to evolve an Azure-based data platform, with a mix of greenfield work and enhancements to an established system. The team embraces AI tools like Copilot and Azure AI Foundry, and the role requires hands-on data engineering across pipelines, data models and integrations.

The ideal candidate has strong Python/PySpark and SQL skills, solid data governance understanding, and the ability to mentor peers in a security-conscious, regulated