Data Engineering Lead – AWS & AI - Manchester

Adria Solutions Ltd; Manchester; Hybrid; GBP 90,000 - 120,000; Full time

Job summary

Adria Solutions Ltd in Manchester is seeking a Data Engineering Lead to own and evolve a modern data platform that supports BI, Data Science and AI initiatives. You will lead a small Data Engineering team, drive architectural decisions, and collaborate with stakeholders to implement scalable batch and real-time processing on AWS, with strong governance and cost control.

Hybrid working pattern in Manchester; you will work closely with technical leaders and senior stakeholders to shape data

Qualifications

• Batch processing, data warehousing, ELT and orchestration.
• Strong SQL and Python development skills.
• Infrastructure as code using Terraform.
• CI/CD, automated testing and modern engineering practices.
• Power BI administration and semantic modelling.

Responsibilities

• Own the reliability, security and evolution of the data platform.
• Modernise SQL Server, SSIS/SSDT and Power BI environment.
• Develop scalable batch and real-time data-processing capabilities.
• Lead design of a Customer Data Platform and AWS lakehouse.
• Govern semantic models, metrics and data definitions.
• Ensure secure self-service analytics for BI, DS and AI teams.
• Automate manual data processes and govern costs.
• Mentor Data Engineers and collaborate with senior stakeholders.

Skills

Batch processing

Data warehousing

ELT orchestration

SQL development

Python development

CI/CD practices

Data governance

Tools

Terraform

Power BI

AWS data services

Iceberg lakehouse

dbt tests

Great Expectations

Job description

Data Engineering Lead – AWS & AI - Manchester

An established financial services organisation is looking for a hands‑on Data Engineering Lead to take ownership of its existing data estate and deliver a modern platform supporting Business Intelligence, Data Science and Applied AI.

This is an influential position combining technical leadership, platform ownership and line management. You’ll modernise the organisation’s data capabilities, introduce scalable batch and real‑time processing, and eliminate manual or fragile processes through intelligent automation. You’ll lead a small team of Data Engineers while remaining closely involved in architecture, technology selection and engineering delivery.

What you’ll be doing

• Own the reliability, security and evolution of the organisation’s data platform.
• Modernise an existing SQL Server, SSIS/SSDT and Power BI environment.
• Develop scalable batch and real‑time data‑processing capabilities.
• Lead the design of a Customer Data Platform.
• Build and evolve a modern AWS lakehouse architecture.
• Create governed semantic models, consistent metrics and clear data definitions.
• Enable secure, self‑service data analysis through modern AI tools.
• Automate manual, repetitive and fragile data processes.
• Support BI, Data Science, Machine Learning and Applied AI teams through a unified platform.
• Own data governance, platform security, monitoring and operating costs.
• Manage, develop and help grow the Data Engineering team.
• Work closely with technical leaders and senior stakeholders across the organisation.

What we’re looking for

You’ll be an experienced Data Engineer or Data Engineering Lead who has built and operated complete data platforms—not simply individual components. You’ll combine strong technical knowledge with the confidence to make architectural decisions, lead change and determine when existing technology should be evolved or replaced.

Your experience should include:

• Batch processing, data warehousing, ELT and orchestration.
• Modern streaming technologies such as Flink or comparable platforms.
• Production AWS data engineering.
• AWS services including S3, Lambda, Glue, Athena, Kinesis, RDS, IAM and CloudWatch.
• Lakehouse architectures and open table formats such as Iceberg.
• Medallion‑style data layering and dimensional modelling.
• Strong SQL and Python development skills.
• Infrastructure as code using Terraform.
• CI/CD, automated testing and modern engineering practices.
• SQL Server, SSIS/SSDT and Power BI.
• Semantic models, shared metrics and data glossaries.
• Data governance, retention, lineage, classification and auditability.
• Managing, mentoring or developing Data Engineers.

You should also be comfortable using advanced AI coding assistants, such as Claude Code or equivalent tools, to accelerate development across code, infrastructure, testing and documentation while maintaining appropriate quality and security controls.

Desirable experience

• Financial services or another regulated environment.
• PySpark for large‑scale batch or streaming transformations.
• Migrating SQL Server and SSIS estates to lakehouse or metadata‑driven architectures.
• Customer Data Platforms or event‑driven systems.
• Data‑quality tools such as dbt tests or Great Expectations.
• MCP servers or governed natural‑language data interfaces.
• Financial crime, fraud or regulatory‑reporting data.
• Power BI administration and semantic modelling.

Why apply?

This is an opportunity to take genuine ownership of a business‑critical data platform and shape how data and AI are used across an organisation. You’ll have the freedom to introduce modern technologies, develop a talented engineering team and build a platform capable of supporting future analytics, AI products and strategic decision‑making.

Location: Manchester

Working pattern: Hybrid