Data Engineering Lead – AWS & AI - Manchester

Adria Solutions; Stretford; Hybrid; GBP 90,000 - 130,000; Full time

Job summary

Adria Solutions is supporting an established financial services client in Manchester to recruit a hands-on Data Engineering Lead who will own the data platform and drive modern AWS-based lakehouse, BI, and AI capabilities.

You will lead a small team, set architectural direction, accelerate data processing with batch and streaming pipelines, and partner with BI, Data Science and compliance teams to deliver scalable, governed data assets.

Qualifications

• Experience building and operating complete data platforms, not just components.
• Ability to make architectural decisions and lead technology change.
• Strong SQL and Python development skills.
• Experience with batch processing, data warehousing and ELT.

Responsibilities

• Own the reliability, security and evolution of the data platform.
• Modernise SQL Server/SSIS/SSDT and Power BI environments.
• Develop scalable batch and real-time data-processing capabilities.
• Lead design of a Customer Data Platform and AWS lakehouse architecture.
• Create governed semantic models and clear data definitions.
• Enable secure, self-service data analysis with AI tools.
• Automate manual data processes and support BI, Data Science, ML teams.
• Manage data governance, monitoring and costs.
• Develop and grow the Data Engineering team.
• Collaborate with senior stakeholders to drive architecture.

Skills

Batch processing

Data warehousing

ELT

Orchestration

Flink

Python

SQL

Terraform

CI/CD

Power BI

AWS

Kinesis

Tools

SQL Server

SSIS/SSDT

Power BI

Terraform

AWS

S3

Glue

Athena

Kinesis

CloudWatch

Job description

Data Engineering Lead – AWS & AI - Manchester

An established financial services organisation is looking for a hands-on Data Engineering Lead to take ownership of its existing data estate and deliver a modern platform supporting Business Intelligence, Data Science and Applied AI.

This is an influential position combining technical leadership, platform ownership and line management. You’ll modernise the organisation’s data capabilities, introduce scalable batch and real-time processing, and eliminate manual or fragile processes through intelligent automation.

You’ll lead a small team of Data Engineers while remaining closely involved in architecture, technology selection and engineering delivery.

What you’ll be doing

• Own the reliability, security and evolution of the organisation’s data platform.
• Modernise an existing SQL Server, SSIS/SSDT and Power BI environment.
• Develop scalable batch and real-time data-processing capabilities.
• Lead the design of a Customer Data Platform.
• Build and evolve a modern AWS lakehouse architecture.
• Create governed semantic models, consistent metrics and clear data definitions.
• Enable secure, self-service data analysis through modern AI tools.
• Automate manual, repetitive and fragile data processes.
• Support BI, Data Science, Machine Learning and Applied AI teams through a unified platform.
• Own data governance, platform security, monitoring and operating costs.
• Manage, develop and help grow the Data Engineering team.
• Work closely with technical leaders and senior stakeholders across the organisation.

What we’re looking for

You’ll be an experienced Data Engineer or Data Engineering Lead who has built and operated complete data platforms—not simply individual components.

You’ll combine strong technical knowledge with the confidence to make architectural decisions, lead change and determine when existing technology should be evolved or replaced.

Your experience should include:

• Batch processing, data warehousing, ELT and orchestration.
• Modern streaming technologies such as Flink or comparable platforms.
• Production AWS data engineering.
• AWS services including S3, Lambda, Glue, Athena, Kinesis, RDS, IAM and CloudWatch.
• Lakehouse architectures and open table formats such as Iceberg.
• Medallion-style data layering and dimensional modelling.
• Strong SQL and Python development skills.
• Infrastructure as code using Terraform.
• CI/CD, automated testing and modern engineering practices.
• SQL Server, SSIS/SSDT and Power BI.
• Semantic models, shared metrics and data glossaries.
• Data governance, retention, lineage, classification and auditability.
• Managing, mentoring or developing Data Engineers.

You should also be comfortable using advanced AI coding assistants, such as Claude Code or equivalent tools, to accelerate development across code, infrastructure, testing and documentation while maintaining appropriate quality and security controls.

Desirable experience

• Financial services or another regulated environment.
• PySpark for large-scale batch or streaming transformations.
• Migrating SQL Server and SSIS estates to lakehouse or metadata-driven architectures.
• Customer Data Platforms or event-driven systems.
• Data-quality tools such as dbt tests or Great Expectations.
• MCP servers or governed natural-language data interfaces.
• Financial crime, fraud or regulatory-reporting data.
• Power BI administration and semantic modelling.

Why apply?

This is an opportunity to take genuine ownership of a business-critical data platform and shape how data and AI are used across an organisation.

You’ll have the freedom to introduce modern technologies, develop a talented engineering team and build a platform capable of supporting future analytics, AI products and strategic decision-making.

Location: Manchester

Working pattern: Hybrid

Data Engineering Lead – AWS & AI - Manchester